Computing architecture for multiple search bots and behavior bots and related devices and methods

ABSTRACT

The amount and variety of data being generated is becoming too extreme for many computing systems to process, and is even more difficult for information systems to provide relevant data to users. A distributed computing system is provided that includes server machines that form a data enablement platform. The platform includes: a plurality of data collectors that stream data over a message bus to a streaming analytics and machine learning engine; a data lake and a massive indexing repository for respectively storing and indexing data; a behavioral analytics and machine learning module; and multiple application programming interfaces (APIs) to interact with the data lake and the massive indexing repository, and to interact with multiple applications. The multiple applications are command cards, and each command card includes a directive module, a memory module, search bots, and behavior bots that operate at least within the data enablement platform.

CROSS-REFERENCE TO RELATED APPLICATION

This patent application claims priority to U.S. Provisional PatentApplication No. 62/548,173, filed on Aug. 21, 2017, and titled“Computing Architecture For Multiple Search Bots And Behavior Bots AndRelated Devices And Methods”, the entire contents of which are herebyincorporated by reference.

TECHNICAL FIELD

In one aspect, the following generally relates to computingarchitectures for multiple search bots and behavior bots to processlarge volumes of data from many different data sources. In anotheraspect, the following generally relates to user devices and relatedcomputing architectures and methods for processing the data andoutputting user interface content, including but not limited to audiocontent or visual content, or both.

DESCRIPTION OF THE RELATED ART

Our information age is growing more and more complex. In the field of“big data”, there is an increasing volume of data, an increasing varietyof data, and an increasing velocity of data. Using most computingsystems, there is too much data to search and too much data is beingcreated for computing systems to keep up.

The amount and variety of data sources are also growing too. There isdata being generated by Internet users, by social sites, by companies,by 3^(rd) party data companies, by Internet of Things (IoT) devices, andby enterprise systems. Even within the category of social media data,there are many types of data formats, many sources of data, and manydifferent meanings of the data. Even within the category of data fromIoT devices, there are numerous types of data formats, many sources ofdata, and many different meanings of the data. This enormous complexityof data, which will continue to grow, leads to a state of “extreme data”(XD).

Web crawler technology, also called Internet bots or search engine bots,indexes data from various websites. However, this technology is focusedon web site data sources and does not take into account extreme data.

It is understood that information systems for machines or for people, orboth, are designed to process data to automatically execute actions orto recommend actions. However, these information systems do not have thehardware resources or software resources, or both, to process extremedata.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments will now be described by way of example only with referenceto the appended drawings wherein:

FIG. 1 is a schematic diagram of an example computing architecture foringesting user data via user devices, and providing big datacomputations and machine learning using a data enablement platform.

FIG. 2 is another schematic diagram, show another representation of thecomputing architecture in FIG. 1.

FIG. 3 is a schematic diagram of oral communication devices (OCDs) incommunication with respective user devices, which are in turn incommunication with the data enablement platform.

FIG. 4A is a schematic diagram showing an OCD being used in a meetingand showing the data connections between various devices and the dataenablement platform.

FIG. 4B is a schematic diagram showing different embodiments of an OCD,including wearable devices, and an OCD embodiment configured to provideaugmented reality or virtual reality.

FIG. 5 is a block diagram showing example components of the OCD.

FIG. 6 is a schematic diagram showing an example software architecturefor command cards.

FIG. 7 is a flow diagram showing the example flow of data between thesoftware components of a given command card.

FIGS. 8 and 9 are schematic diagrams showing an example computingarchitecture for a data enablement platform, which includes supportingparallelized search bots and parallelized behavior bots.

FIGS. 10A and 10B are block diagram showing example softwarearchitectures used to coordinate multiple command cards using a teamcommand card.

FIG. 11 is a block diagram of example software modules residing on theuser device and the data enablement platform.

FIGS. 12 and 13 are screenshots of example graphical user interfaces(GUIs) of a command card displayed on a user device.

FIG. 14 is a flow diagram of example executable instructions for using acommand card with the data enablement platform to provide relevantinformation for a given directive.

FIG. 15 is another flow diagram showing the flow of data between searchbots and behavior bots.

FIG. 16 is a flow diagram of example executable instructions for usingthe data enablement platform to monitor a given topic.

FIG. 17 is a flow diagram of example executable instructions for usingthe data enablement platform to monitor a given topic, including usingboth internal and external data.

FIG. 18 is a flow diagram of example executable instructions for usingthe data enablement platform to modify the audio parameters of certainphrases and sentences.

FIG. 19 is a flow diagram of example executable instructions for usingthe data enablement platform to extract data features from voice dataand associated background noise.

FIG. 20 is a flow diagram of example executable instructions for usingthe data enablement platform to provide meaningful and interactive datato the user device.

FIG. 21 is an example embodiment of a Digital Signal Processing(DSP)-based voice synthesizer.

FIG. 22 is an example embodiment of a hardware system used by theDSP-based voice synthesizer.

DETAILED DESCRIPTION

It will be appreciated that for simplicity and clarity of illustration,where considered appropriate, reference numerals may be repeated amongthe figures to indicate corresponding or analogous elements. Inaddition, numerous specific details are set forth in order to provide athorough understanding of the example embodiments described herein.However, it will be understood by those of ordinary skill in the artthat the example embodiments described herein may be practiced withoutthese specific details. In other instances, well-known methods,procedures and components have not been described in detail so as not toobscure the example embodiments described herein. Also, the descriptionis not to be considered as limiting the scope of the example embodimentsdescribed herein.

It is herein recognized that it is desirable to process data to providerelevant or meaningful results. It is recognized that there are alsomany technical challenges of existing computing systems.

In particular, it is herein recognized that typical computingarchitectures and software programs for information systems are limitedto ingest limited types of data and usually from a small number of datasources. Typically, these types of data are based on internal databases.However, it is herein recognized that there are many more types of data,and from different data sources, that can be used and processed toprovide interesting data to a person. It is recognized that utilizingextreme data could potentially provide more relevant data. For example,it is recognized that data sources can include, but are not limited to,any one or more of: data from IoT devices, various newspaper servers,various enterprise software systems, various television channels,various radio networks, various magazine servers, social data networksand related platforms, internal databases, data obtained via individualuser devices, stock exchange platforms, blogs, third-party searchengines, etc. From these example sources, it is appreciated that thetypes of data are varied and that the data can be constantly updating.

It is also recognized that typical information systems are nottechnically suited to process extreme data effectively, or, in otherwords, are designed to typically to process data from a select fewsources of data. One of the technical challenges of processing extremedata include being able to process many different types of data and inrealtime, or near realtime. Another technical challenge is that storingand organizing the extreme data requires vast memory resources andprocessing resources (e.g. both hardware and software).

Furthermore, it is herein recognized that the possession of extreme datain itself does not provide relevant or meaningful information, which isthe purpose of an information system. It is herein recognized that it isdifficult to identify meaningful data from the obtained extreme data.The technical challenge is further complicated when different users wishto obtain different meaningful information from the information system.For example, a first user wishes to use the information system to obtainmeaningful information about a first thing, or feature, or person, orconcept, etc., while a second user tries to use the same informationsystem to obtain meaningful information about a second thing, orfeature, or person, or concept, etc. The more divergent the types ofinformation being sought out by the first user and the second user, themore technically challenging it is to build an effective informationsystem to accommodate these types of information. As a result,currently, many organizations use and rely on many different informationsystems, with each information system typically focused on one or twotypes of information. The technical challenge to build an informationsystem further grows in difficulty with a very large number of users.

It is herein recognized that, not only are current computingarchitectures insufficient to process extreme data, current front-endcomputing systems (e.g. user interface software and devices) are alsolimited. For example, in many current information systems, a usertypically types in text data into predefined fields, such as via aphysical keyboard or a touchscreen keypad. These predefined input fieldsand input GUIs are processed using more typical computing software. Itis herein recognized that such an approach inherently ignores utilizingthe variety and the volume of data that is available for various datasources, which likely have data types and data formats that do notconform to the predefined input forms and input GUIs.

It is herein recognized that people often think, talk and act innon-predefined patterns. In other words, the thought process of aperson, or a conversation between people, does not typically followpredefined GUIs and predefined input forms. Using existing GUIs, aperson will need to extract their notes from a conversation and inputthe extracted portions of information into the predefined GUIs and inputforms. This process is even more burdensome and complex when many peoplehave a meeting, and a person must identify the relevant information totype into a predefined GUI or predefined input forms. Not only is thisdata entry process inefficient, but the technology inherently ignoresother data from the individual's thoughts, or the conversations, or themeetings, or combinations thereof. In other words, current informationtechnologies do not recognize the value of the user behavior. Nor docurrent information technologies collect data relating to user behavior,process the same, and apply the understood behaviors to the overallinformation system.

Furthermore, it is herein recognized that using current searchinterfaces for users, people need to spend time and mental effort toinput correct search terms into the predefined GUIs provided by theinformation systems, analyze the results, and to make a decision aboutthe analyzed results. It is herein recognized that information systemswould be more effective if they could provide a user with the answer oranswers to a user's question.

Therefore, computing architectures, computing functionalities anddevices are described herein to address one or more of the abovetechnical challenges.

In an example embodiment, a computing system is provided that includes adata enablement platform that comprises multiple computing machines thatare in communication with each other over a data network. The computingsystem also include a user device that includes a user interface toreceive user input to generate a command card. Associated with thecommand card includes data representing a directive, multiple searchbots, multiple behavior bots, a memory module, and a user interfacemodule. The multiple search bots reside on the data enablement platform.Each one of the search bots search for data specific to the samedirective of the command card, but each search bot is also specific to adifferent data source. In other words, a first search bot searches afirst data source and a second search bot searches a second data source.The search bots use a distributed streaming analytics computing platformto process the data from their respective data sources. The first searchbot, for example, uses a different set of computations to process thedata from the first data source, compared to the second search botassociated with the second data source. The results from the search botsare then processed by the behavior bots.

The behavior bots also reside on the data enablement platform andinteract with the user interface module of the command card. Thebehavior bots receive user interaction data from the user interfacemodule, and process the same using data science and machine learning, toidentify different behavior attributes. The behavior bots process theresults outputted by the search bots to produce personalized dataresults. For example, the behavior bots filter the results outputted bythe search bots to reflect the behavior attributes. In another example,either in addition or in alternative, the behavior bots transform theresults outputted by the search bots to reflect the behavior attributes.

In an example aspect, the behavior bots include artificial constraintsthat reflect the behavior attributes of the user. For example, a userhas a do-it-yourself personality, which is captured by the user'sbehavior bot. When the search bots search for furniture (e.g. adirective provided by the user), then the behavior bot automaticallysurfaces results with furniture that has a do-it-yourself aspect (e.g.assembly is required, modifications can be made to the furniture, thefurniture looks hand-crafted, etc.).

In another example aspect, the behavior bots add data (e.g. meta data)that is reflective of the behavior attributes of the user. For example,a user has an affinity to cars, which is captured by the user's behaviorbot. The search bots search for relationship advice (e.g. dating,marriage, etc.) according a directive provided by the user, and thesearch bots obtain relationship advice. The behavior bot automaticallyadds and combines car analogies and other data (e.g. images of cars,video clips of cars) with the relationship advice data. In this way, theuser can more comfortably understand the relationship advice data as ithas been augmented by car analogies and other car-related data.

In another example aspect, the behavior bots augment the data in a waythat is reflective of the behavior attributes of the user. For example,a user is risk adverse, which is captured by the user's behavior bot.The search bots search for vacation destinations according to adirective by the user, and the search bots identify several vacationlocations that meet the user's setout criteria (e.g. general location,budget, beach, etc.). The user's behavior bot automatically identifiespotential risks and safety measures associated with each vacationlocation and ranks the vacation locations from least risky to mostrisky. For example, the behavior bot identifies the number and severityof risks, such as war, theft, civil unrest, health risk, naturaldisaster risk (e.g. volcano, hurricane, fire, etc.), that are associatedwith the different vacation locations and accordingly ranks the vacationlocations. In another example aspect, the behavior bot identifies therisks and the safety measures to be taken to counter the risks (e.g.travel vaccines, travel routes, etc.).

In another example embodiment, the behavior bots augment thepresentation of the data in a way that is reflective of the behaviorattributes of the user. For example, if the behavior bot detects that auser has a sad mood, then the behavior bot reads aloud (e.g. play anaudio file) of a news article in a cheerful voice. For example, if thebehavior bot detects that a user has a busy or concentrated mood, thenthe behavior bot reads aloud (e.g. play an audio file) of a news articlein a fast-paced and neutral voice.

It can be appreciated that the behavior bots can modify the search datain various ways to reflect the behaviors of a user.

The results are outputted by the behavior bots are stored in associationwith the command card and are outputted to the user. In an exampleembodiment, a summarization of the results outputted by the behaviorbots are stored in association with the command card with data links tovarious data sources, which were obtained by the search bots. The dataenablement platform includes a data lake that stores this information.

The results outputted by the behavior bots are presented to the user viaa user interface. As the user interacts with the user interface, thebehavior bots receive data reflecting the user behavior. The behaviorbots use this data to repeatedly update parameters used in the datascience and machine learning computations, or to automatically selectnew data science algorithms or machine learning algorithms, or acombination thereof.

It is herein recognized that meaningful data for a person is subjectiveand reflective of the person's behavior. Furthermore, a person makes ordraws relationships and conclusions between two different ideas (e.g.two different topics, two different data sets, etc.) that tends to bereflective of their behavior. The term behavior herein includes aperson's habits, emotions, opinions, personality, desires, thoughts,pre-conceived notions, biases, perspectives, etc. Different behaviorbots capture different aspects of a person's behavior (e.g. by capturingbaseline data of a person, by making assumptions, by using look-alikealgorithms, etc.) and take into account the person's behavioralattributes when conducting searches for relevant data. In an exampleaspect, this leads to finding and providing more meaningful data fromthe perspective of the person.

The behavior bots detect and infer behavioral attributes of a person. Insome aspects, a person has difficulty realizing or identifying their ownbehavioral attributes, but the behavior bots are able to use datacollected over a period of time about the person and data collected fromothers to identify those behavioral attributes.

In another example aspect, the combination of behavior bots and searchbots allows for a person's directive to obtain relevant and personalizedanswers to be scaled. In other words, the behavior bots and search botscan be used to search and process many different types of data for manydifferent topics, and from many data sources, while still taking intoaccount the person's behavior.

In a further example aspect, the behavior bots are transferrable. Forexample, a given behavior bot used in a search for a first topic canalso be used in a search for a second topic. The second topic could beunrelated to the first topic, for example. In another example, a givenbehavior bot associated with a first person is transferred to a secondperson, so that the second person's data searching and processing takeson the same behavioral qualities as the first person's data searchingand processing.

In an example embodiment, a user interface presents multiple commandcards, each associated with a different directive. Each command card isassociated with the multiple search bots and the multiple behavior bots,which are specific to the directive of the command card.

It will be appreciated that the term “bot” is known in computingmachinery and intelligence to mean a software robot or a software agent.The bots described herein, such as the search bots and the behaviorbots, have artificial intelligence.

It will also be appreciated that a user can interact with the userinterface in various ways, including without limitation: using orallanguage, visually, using gestures, using facial expressions, by typing,by using selections presented in a GUI, by using brain signals, by usingmuscle signals, by using nerve signals, or a combination thereof.

In an example embodiment, an oral communication user device (e.g. adevice that includes a microphone) records oral information from a user(e.g. the user's word and sounds) to interact with a data enablementsystem. The data enablement system processes the voice data to extract,at least the words and of the spoken language, and accordingly processesthe data using artificial intelligence computing software and datascience algorithms. The data obtained from the oral communication deviceis processed in combination with, or comparison with, or both, internaldata specific to an organization and external data. The computingarchitecture, via the multiple search bots, ingests data from externaldata sources or internal data sources, or both, to provide real-timeoutputs or near real-time data outputs, or both. The data outputs arepresented to the user as audio feedback, or visual feedback, or both.Other types of user feedback may be used, including tactile feedback.Other machine actions may be initiated or executed based on the dataoutputs.

In another example embodiment, the oral communication device is awearable technology that tracks a user's movement. Currently known andfuture known wearable devices are applicable to the principles describedherein. In another example embodiment, the oral communication device ispart of a virtual reality system or augmented reality system, or both.In other words, the display of visual data is immersive and the user caninteract with the visual data using oral statements and questions, orusing physical movement, or using facial expressions, or a combinationthereof.

Turning to FIG. 1, a user device 102 interacts with a user 101. Theuser, for example, is a consumer user, a business user, a user in thesales industry, a user in the manufacturing industry, etc. The userdevice 102 includes, amongst other things, input devices 113 and outputdevices 114. The input devices include, for example, a microphone andkeyboard (e.g. physical keyboard or touchscreen keyboard, or both).Other types of input devices include brain signal sensors, nerve signalsensors or muscle signal sensors, or a combination thereof, which can beused to detect the speech, thoughts or intentions (or a combinationthereof) of the user. The output devices include, for example, an audiospeaker and a display screen. Non-limiting examples of user devicesinclude a mobile phone, a smart phone, a tablet, smart watches, headsetsthat provide augmented reality or virtual reality or both, a desktopcomputer, a laptop, an e-book, a wearable device, and an in-car computerinterface. In another example, a system of user devices is provided togenerate a visually immersive user interface, so that a person's voiceis being recorded, and their gestures and motions are being tracked tointeract with the data. The user device, or user devices, are incommunication with a 3^(rd) party cloud computing service 103, whichtypically includes banks of server machines. Multiple user devices 111,which correspond to multiple users 112, can communicate with the 3^(rd)part cloud computing service 103.

The cloud computing service 103 is in data communication with one ormore data science server machines 104. These one or more data scienceserver machines are in communication with internal application anddatabases 105, which can reside on separate server machines, or, inanother example embodiment, on the data science server machines. In anexample embodiment, the data science computations executed by the datascience servers and the internal applications and the internal databasesare considered proprietary to given organization or company, andtherefore are protected by a firewall 106. Currently known firewallhardware and software systems, as well as future known firewall systemscan be used.

The data science server machines, also called data science servers, 104are in communication with an artificial intelligence (AI) platform 107.The AI platform 107 includes one or more AI application programminginterfaces (APIs) 108 and an AI extreme data (XD) platform 109. As willbe discussed later, the AI platform runs different types of machinelearning algorithms suited for different functions, and these algorithmscan be utilized and accessed by the data science servers 104 via an AIAPI.

The AI platform also is connected to various data sources 110, which maybe 3^(rd) party data sources or internal data sources, or both.Non-limiting examples of these various data sources include: newsservers, radio networks, television channel networks, magazine servers,stock exchange servers, IoT data, enterprise databases, social mediadata, media databases, etc. In an example embodiment, the AI XD platform109 ingests and processes the different types of data from the variousdata sources.

In an example embodiment, the network of the servers 103, 104, 105, 106,107 and optionally 110 make up a data enablement system. The dataenablement system provides relevant to data to the user devices, amongstother things. In an example embodiment, all of the servers 103, 104,105, 106 and 107 reside on cloud servers.

An example of operations is provided with respect to FIG. 1, using thealphabetic references. At operation A, the user device 102 receivesinput from the user 101. For example, the user is speaking and the userdevice records the audio data (e.g. voice data) from the user. The usercould be providing a command or a query to the data enablement system,which is used to generate a directive of a command card. The user couldalso use the user device to make an audio recording or to captureimages, or both, for memorializing thoughts to himself or herself, orproviding himself or herself a to-do list to complete in the future. Inanother example embodiment, the user device includes or is in datacommunication with one or more sensors that detect brain signals, musclesignals or nerve signals, or a combination thereof, which are used asuser input data. For example, the brain signals, muscle signals, ornerve signals, or a combination thereof, are processed to generatespeech data of the user. In other words, while many of the examplesherein refer to audio data or voice data being the user input data, itwill be appreciated that in similar example embodiments, other types ofuser input data can be used.

In an example embodiment, a data enablement application is activated onthe user device and this application is placed into a certain mode,either by the user or autonomously according to certain conditions. Forexample, this certain mode is specific to a given command card. Forexample, if a user is associated with a first command card, a secondcommand card, a third command card, and so forth, then the user'sselection of the second command card places the data enablementapplication into the mode of the second command card.

At operation B, the user device transmits the recorded audio data to the3^(rd) party cloud computing servers 103. In an example embodiment, theuser device also transmits other data to the servers 103, such ascontextual data (e.g. time that the message was recorded, informationabout the user, the mode of the data enablement application during whichthe message was recorded, etc.). These servers 103 apply machineintelligence, including artificial intelligence, to extract datafeatures from the audio data. These data features include, amongst otherthings: text, sentiment, emotion, background noise, a command or query,or metadata regarding the storage or usage, or both, of the recordeddata, or combinations thereof.

At operation C, the servers 103 send the extracted data features and thecontextual data to the data science servers 104. In an exampleembodiment, the servers 103 also send the original recorded audio datato the data science servers 104 for additional processing.

At operation D, the data science servers 104 interact with the internalapplications and databases 105 to process the received data. Inparticular, the data science servers store and executed one or morevarious data science algorithms to process the received data (fromoperation C), which may include processing proprietary data andalgorithms obtained from the internal applications and the databases105.

In alternative, or in addition to operation D, the data science servers104 interact with the AI platform 107 at operations E and G. In anexample embodiment, the data science servers 104 have algorithms thatprocess the received data, and these algorithms transmit information tothe AI platform for processing (e.g. operation E). The informationtransmitted to the AI platform can include: a portion or all of the datareceived by the data science servers at operation C; data obtained frominternal applications and databases at operation D; results obtained bythe data science servers from processing the received data at operationC, or processing the received data at operation D, or both; or acombination thereof. In turn, the AI platform 107 processes the datareceived at operation E, which includes processing the informationingested from various data sources 110 at operation F. Each of thesearch bots operate on the AI platform 107 and the data science servers104 to search for and obtain information that is relevant to their givendirective. Subsequently, the AI platform 107 returns the results of itsAI processing to the data science servers in operation G.

Based on the results received by the data science servers 104 atoperation G, the data science servers 104, for example, update itsinternal applications and databases 105 (operation D) or its own memoryand data science algorithms, or both. The data science servers 104 alsoprovide an output of information to the 3^(rd) party cloud computingservers 104 at operation H. This outputted information may be a directreply to a query initiated by the user at operation A. In anotherexample, either in alternative or in addition, this outputtedinformation may include ancillary information that is eitherintentionally or unintentionally requested based on the received audioinformation at operation A. In another example, either in alternative orin addition, this outputted information includes one or more commandsthat are either intentionally or unintentionally initiated by receivedaudio information at operation A. These one or more commands, forexample, affect the operation or the function of the user device 102, orother user devices 111, or IoT devices in communication with the 3^(rd)party cloud computing servers 104, or a combination thereof.

The behavior bots, or portions thereof, reside on the AI platform, orthe data science servers, or the 3^(rd) party cloud computing servers,or a combination thereof. Portions of the behavior bots, for example,also reside on user devices, smart devices, phones, edge devices, andIoT devices. In other words, the behavior bots reside onuser-interactive devices or devices that are in proximity touser-interactive devices, and are actively monitoring user interactiondata. After from the search bots are obtained, the behavior bots processthis data to present personalized data to the user.

The 3^(rd) party cloud computing servers 104, for example, takes thedata received at operation H and applies transformation to the data, sothat the transformed data is suitable for output at the user device 102.For example, the servers 104 receive text data at operation H, and thenthe servers 104 transform the text data to spoken audio data. Thisspoken audio data is transmitted to the user device 102 at operation I,and the user device 102 then plays or outputs the audio data to the userat operation J.

This process is repeated for various other users 112 and their userdevices 111. For example, another user speaks into another user deviceat operation K, and this audio data is passed into the data enablementplatform at operation L. The audio data is processed, and audio responsedata is received by the another user device at operation M. This audioresponse data is played or outputted by the another user device atoperation N.

In another example embodiment, the user uses touchscreen gestures,movements, typing, brain signals, muscle signals, nerve signals, etc. toprovide inputs into the user device 102 at operation A, either inaddition or in alternative to the oral input. In another exampleembodiment, the user device 102 provides visual information (e.g. text,video, pictures) either in addition or in alternative to the audiofeedback at operation J.

Turning to FIG. 2, another example of the servers and the devices areshown in a different data networking configuration. The user device 102,the cloud computing servers 103, the data science servers 104, AIcomputing platform 107, and the various data sources 110 are able totransmit and receive data via a network 201, such as the Internet. In anexample embodiment, the data science servers 104 and the internalapplications and databases 105 are in communication with each other overa private network for enhanced data security. In another exampleembodiment, the servers 104 and the internal applications and thedatabases 105 are in communication with each other over the same network201.

As shown in FIG. 2, example components of the user device 102 include amicrophone, one or more other sensors, audio speakers, a memory device,one or more display devices, a communication device, and one or moreprocessors. The user device can also include a global positioning systemmodule to track the user device's location coordinates. This locationinformation can be used to provide contextual data when the user isconsuming digital content, or interacting with the digital content (e.g.adding notes, swipe gestures, eye-gaze gestures, voice data, addingimages, adding links, sharing content, providing brain signals,providing muscle signals, providing nerve signals, etc.), or both.

In an example embodiment, the user device's memory includes various userinterface bots that are part of the data enablement application, whichcan also reside on the user device. These bots include processing thatalso resides on the 3^(rd) party cloud computing servers 103. These userinterface bots have chat capabilities. Examples of chat bot technologiesthat can be adapted to the system described herein include, but are notlimited to, the trade names Siri, Google Assistant, Alexa, and Cortana.In an example aspect, the bot used herein has various languagedictionaries that are focused on various topics. In an example aspect,the bot used herein is configured to understand questions and answersspecific to various topics. The user interface bots described herein notlimited to chatting capabilities, but can also include functionality andartificial intelligence for controlling visual display of information(e.g. images and video) in combinations with language.

In an example aspect, one or more behavior bots used herein learn theunique voice of the user, which the one or more behavior botsconsequently uses to learn behavior that may be specific to the user.This anticipated behavior in turn is used by the data enablement systemto anticipate future questions and answers related to a given topic.This identified behavior is also used, for example, to make actionrecommendations to help the user achieve a result, and these actionrecommendations are based on the identified behaviors (e.g. identifiedvia machine learning) of higher ranked users having the same topicinterest. For example, users can be ranked based on their expertise on atopic, their influence on a topic, their depth of commentary (e.g.private commentary or public commentary, or both) on a topic, thecomplexity of their bots for a given topic, etc.

In an example aspect, a given behavior bot applies machine learning toidentify unique data features in the user voice. Machine learning caninclude, deep learning. Currently known and future known algorithms forextracting voice features are applicable to the principles describedherein. Non-limiting examples of voice data features include one or moreof: tone, frequency (e.g. also called timbre), loudness, rate at which aword or phrase is said (e.g. also called tempo), phonetic pronunciation,lexicon (e.g. choice of words), syntax (e.g. choice of sentencestructure), articulation (e.g. clarity of pronounciation), rhythm (e.g.patterns of long and short syllables), and melody (e.g. ups and downs invoice). As noted above, these data features can be used identifybehaviors and meanings of the user, and to predict the content, behaviorand meaning of the user in the future. It will be appreciated thatprediction operations in machine learning include computing data valuesthat represent certain predicted features (e.g. related to content,behavior, meaning, action, etc.) with corresponding likelihood values.

The user device may additionally or alternatively receive video data orimage data, or both, from the user, and transmit this data via a bot tothe data enablement platform. The data enablement platform is thereforeconfigured to apply different types of machine learning to extract datafeatures from different types of received data. For example, the 3^(rd)party cloud computing servers use natural language processing (NLP)algorithms or deep neural networks, or both, to process voice and textdata. In another example, the 3^(rd) party cloud computing servers usemachine vision, or deep neural networks, or both, to process video andimage data.

Turning to FIG. 3, an example embodiment of an oral communication device(OCD) 301 is shown, which operates in combination with the user device102 to reduce the amount of computing resources (e.g. hardware andprocessing resources) that are consumed by the user device 102 toexecute the data enablement functions, as described herein. In somecases, the OCD 301 provides better or additional sensors than a userdevice 102. In some cases, the OCD 301 is equipped with better oradditional output devices compared to the user device 102. For example,the OCD includes one or more microphones, one or more cameras, one ormore audio speakers, and one or more multimedia projects which canproject light onto a surface. The OCD also includes processing devicesand memory that can process the sensed data (e.g. voice data, videodata, etc.) and process data that has been outputted by the dataenablement platform 303. As noted above, the data enablement platform303 includes, for example, the servers 103, 104, 105, and 107.

As shown in FIG. 3, the OCD 301 is in data communication with the userdevice via a wireless or wired data link. In an example embodiment, theuser device 102 and the OCD 301 are in data communication using aBluetooth protocol. The user device 102 is in data communication withthe network 201, which is in turn in communication with the dataenablement platform 303. In operation, when a user speaks or takesvideo, the OCD 301 records the audio data or visual data, or both. TheOCD 301, for example, also pre-processes the recorded data, for example,to extract data features. The pre-processing of the recorded data mayinclude, either in addition or in the alternative, data compression.This processed data or the original data, or both, are transmitted tothe user device 102, and the user device transmits this data to the dataenablement platform 303, via the network 201. The user device 102 mayalso transmit contextual data along with the data obtained or producedby the OCD 301. This contextual data can be generated by the dataenablement application running on the user device 102, or by the OCD301.

Outputs from the data enablement platform 303 are sent to the userdevice 102, which then may or may not transmit the outputs to the OCD301. For example, certain visual data can be displayed directly on thedisplay screen of the user device 102. In another example embodiment,the OCD receives the inputs from the user device and provides the userfeedback (e.g. plays audio data via the speakers, displays visual datavia built-in display screens or built-in media projectors, etc.).

In an example embodiment, the OCD 301 is in data connection with theuser device 102, and the OCD 301 itself has a direct connection to thenetwork 201 to communicate with the data enablement platform 303. Inanother example, the user uses the user device 102 to interact with thedata enablement platform without the OCD.

In an example embodiment, a different example of a silent OCD 304 isused to record the language inputs of the user. The silent OCD 304includes sensors that detects other user inputs, but which are not thevoice. Examples of sensors in the silent OCD 304 include one or more of:brain signal sensors, nerve signal sensors, and muscle signal sensors.These sensors detect silent gestures, thoughts, micro movements, etc.,which are translated to language (e.g. text data). In an exampleembodiment, these sensors include electrodes that touch parts of theface or head of the user. In other words, the user can provide languageinputs without having to speaking into a microphone. The silent OCD 304,for example, is a wearable device that is worn on the head of the user.The silent OCD 304 is also sometimes called a silent speech interface ora brain computer interface. The silent OCD 304, for example, allows auser to interact with their device in a private manner while in a groupsetting (see FIG. 4A) or in public.

Similar functionality is applicable to the other instance of the OCD 301that is in data communication with the desktop computer 302. Inparticular, it is herein recognized that many existing computing devicesand user devices are not equipped with sensors of sufficient quality,nor with processing hardware equipped to efficiently and effectivelyextract the features from the sensed data. Therefore, the OCD 301supplements and augments the hardware and processing capabilities ofthese computing devices and user devices.

Turning to FIG. 4A, the OCD 301 is shown being used in a meeting withvarious people, each having their own respective user devices 401, 402,403, 404, 405, 304. As noted above, the types of user device can bevarious (e.g. wearable technology, headsets with virtual reality and/oraugmented reality capabilities, smart phones, tablets, laptops, etc.).The OCD can also be used to record data (e.g. audio data, visual data,etc.) and provide data to people 406 that do not have their own userdevice. The OCD records the oral conversation of the meeting to, forexample, take meeting notes. In another aspect, the OCD also links tothe user devices to give them information, for example, in real-timeabout the topics being discussed during the meeting. The OCD alsoreduces the computing resources (e.g. hardware and processing resources)on the individual user devices.

In an example embodiment, the user 406 wears a silent OCD 304 toprivately interact using with the OCD 301. For example, the user's brainsignals, nerve signals, muscle signals, or a combination thereof, arecaptured are synthesized into speech. In this way, the user 406 can attimes give private or silent notes, commands, queries, etc. to the OCD301, and at other times, provide public notes, commands, queries, etc.to the OCD 301 that are heard by the other users in the meeting.

In an example embodiment, the user devices 401, 402, 403, 404, 405, 304are in data communication with the OCD 301 via a wireless connection, ora wired connection. In an example embodiment, some of the user devices401, 402 do not have Internet access, but other user devices 403, 404,405 do have Internet access over separate data connections X, Y and Z.Therefore, the OCD 301 uses one or more of these data connections X, Yand Z to transmit and receive data from the data enablement platform303.

The OCD may use different communication routes based on the availablebandwidth, which may be dictated by the user devices.

For example, the OCD parses a set of data to be transmitted to the dataenablement platform into three separate data threads, and transmitsthese threads respectively to the user devices 403, 404 and 405. Inturn, these data threads are transmitted by the user devices over therespective data connections X, Y and Z to the data enablement platform303, which reconstitute the data from the separate threads into theoriginal set of data.

Alternatively, the OCD uses just one of the data connections (e.g. X)and therefore funnels the data through the user device 403.

In another example embodiment, the OCD designates the data connections Xand Y, corresponding to the user deices 403 and 404, for transmittingdata to the data enablement platform 303. The OCD designates the dataconnection Z, corresponding to the user device 405, for receiving datafrom the data enablement platform 303.

The data obtained by the OCD, either originating from a user device orthe data enablement platform, can be distributed amongst the userdevices that are in communication with the OCD. The OCD can also providecentral user feedback (e.g. audio data, visual data, etc.) to the usersin the immediate vicinity.

It will be appreciated that the OCD therefore acts as a local centralinput and output device. In another example aspect, the OCD also acts asa local central processing device to process the sensed data, orprocessed the data from the data enablement platform, or both. Inanother example aspect, OCD also acts as a local central communicationhub.

In an example embodiment, the OCD, either in the alternative or inaddition, the OCD has its own network communication device and transmitsand receives data, via the network 201, with the data enablementplatform 303.

The OCD provides various functions in combination with the dataenablement platform 303. In an example operation, the OCD provides anaudio output that orally communicates the topic of a discussion. In anexample operation, the OCD records the discussion items that are spokenduring a conversation, and automatically creates text containing meetingminutes. In an example operation, the OCD monitors the flow of thediscussion and the current time, and at appropriate times (e.g. pauses,hard stops, end of sentences, etc.) interjects to provide audio feedbackabout related topics. For example, the OCD monitors topics and conceptsbeing discussed and, in real-time, distributes ancillary and relateddata intelligence to the user devices. In an example operation, the OCDmonitors topics and concepts being discussed and, in real-time,determines if pertinent related news or facts are to be shared and, ifso, interjects the conversation by providing audio or visual output thatprovides the pertinent related news or facts. In another exampleoperation, the OCD monitors topics and concepts being discussed and, inreal-time, determines if a user provided incorrect information and, ifso, interjects the conversation by providing audio or visual output thatprovides the correct information. In another example operation, the OCDprovides different feedback to different user devices, to suit theinterests and goals specific the different users, during theconversation between users. In another example operation, the OCD usescameras and microphones to record data to determine the emotion andsentiment of various users, which helps to inform: which content shouldbe published to certain users; the order or format, or both, of thepresentation of the content; and the generation of new content. Inanother example operation, each of the users can use their user devicesin parallel to interact with the OCD or the data enablement platform, orboth, to conduct their own research or make private notes (or both)during the conversation. In another example embodiment, the OCD includesone or more media projectors to project light images on surroundingsurfaces.

It will be appreciated that while the housing body of the OCD is shownto be cylindrical, in other example embodiments, it has differentshapes.

Turning to FIG. 4B, users in Location A are interacting with one or moreOCDs, and a user in a separate location (i.e. Location B) is interactingwith another OCD. Together, these users, although at different locationscan interact with each through digital voice and imagery data. The dataenablement platform processes their data inputs, which can include voicedata, image data, physical gestures and physical movements. These datainputs are then used to by the data enablement platform to providefeedback to the users.

At Location A, two OCD units 301 are in data communication with eachother and project light image areas 411, 410, 409, 408. These projectedlight image areas are positioned in a continuous fashion to provide, ineffect, a single large projected light image area that can surround orare around the users. This produces an augmented reality or virtualreality room. For example, one OCD unit projects light image areas 411and 410, and another OCD unit projects light image areas 409 and 408.

Also at Location A is a user 407 that is wearing another embodiment ofan OCD 301 a. This embodiment of the OCD 301 a includes a microphone,audio speakers, a processor, a communication device, and otherelectronic devices to track gestures and movement of the user. Forexample, these electronic devices include one or more of a gyroscope, anaccelerometer, and a magnetometer. These types of devices are allinertial measurement units, or sensors. However, other types of gestureand movement tracking can be used. In an example embodiment, the OCD 301a is trackable using triangulation computed from radio energy signalsfrom the two OCD units 301 positioned at different locations (but bothwithin Location A). In another example, image tracking from cameras isused track gestures.

The users at Location A can talk and see the user at Location B.

Conversely, the user at Location B is wearing a virtual reality oraugmented reality headset, which is another embodiment of an OCD 301 b,and uses this to talk and see the users at Location A. The OCDembodiment 301 b projects or displays images near the user's eyes, oronto the user's eyes. The OCD embodiment 301 b also includes amicrophone, audio speaker, processor, and communication device, amongstother electronic components. Using the OCD embodiment 301 b, the user isable to see the same images being projected onto one or more of theimage areas 411, 410, 409, and 408.

Turning to FIG. 5, example components that are housed within the OCD 301are shown. The components include one or more central processors 502that exchange data with various other devices, such as sensors 501. Thesensors include, for example, one or more microphones, one or morecameras, a temperature sensor, a magnetometer, one or more inputbuttons, and other sensors.

In an example embodiment, there are multiple microphones that areoriented to face in different directions from each other. In this way,the relative direction or relative position of an audio source can bedetermined. In another example embodiment, there are multiplemicrophones that are tuned or set to record audio waves at differentfrequency ranges (e.g. a microphone for a first frequency range, amicrophone for a second frequency range, a microphone for a thirdfrequency range, etc.). In this way, more definition of audio data canbe recorded across a larger frequency range.

In an example embodiment, there are multiple cameras that are orientedto face in different directions. In this way, the OCD can obtain a 360degree visual field of view. In another example, one or more camerashave a first field of a view with a first resolution and one or morecameras have a second field of view with a second resolution, where thefirst field of view is larger than the second field of view and thefirst resolution is lower than the second resolution. In a furtherexample aspect, the one or more cameras with the second field of viewand the second resolution can be mechanically oriented (e.g. pitched,yawed, etc.) while the one or more cameras with the first field of viewand the first resolution are fixed. In this way, video and images can besimultaneously taken from a larger perspective (e.g. the surroundingarea, people's bodies and their body gestures), and higher resolutionvideo and images can be simultaneously taken for certain areas (e.g.people faces and their facial expressions). It will be appreciated thatcurrently known and future known image processing algorithms and facialexpression data libraries that are used to process facial expressionsare applicable to the principles described herein.

The OCD also includes one or more memory devices 503, lights 505, one ormore audio speakers 506, one or more communication devices 504, one ormore built-in-display screens 507, and one or more media projectors 508.The OCD also includes one or more graphics processing units (GPUs) 509.GPUs or other types of multi-threaded processors are configured forexecuting AI computations, such as neural network computations. The GPUsare also used, for example, to process graphics that are outputted bythe multimedia projector(s) or the display screen(s) 507, or both.

In an example embodiment, the communication devices include one or moredevice-to-device communication transceivers, which can be used tocommunicate with one or more user devices. For example, the OCD includesa Bluetooth transceiver. In another example aspect, the communicationdevices include one or more network communication devices that areconfigured to communicated with the network 201, such as a network cardor WiFi transceiver, or both.

In an example embodiment, there are multiple audio speakers 506positioned on the OCD to face in different directions. In an exampleembodiment, there are multiple audio speakers that are configured toplay sound at different frequency ranges.

In an example embodiment, the built-in display screen forms a curvedsurface around the OCD housing body. In an example embodiment, there aremultiple media projectors that project light in different directions.

In an example embodiment, the OCD is able to locally pre-process voicedata, video data, image data, and other data using on-board hardware andmachine learning algorithms. This reduces the amount of data beingtransmitted to the data enablement platform 303, which reduced bandwidthconsumption. This also reduces the amount of processing required by thedata enablement platform.

Turning to FIG. 6, an example embodiment of computing modules are shownfor two command cards. It will be appreciated that a user may initiatethe creation of different command cards for different topics orrequests.

For example, a user works at a company that sells products. The user maycreate a first command card by speaking into the user device: “How do Iform a relationship with Company A based on the sale of our products?”This is used to form the directive of a Command Card A (e.g. referencedas CC.A in the short form), for example. The same user may type or speakinto the user device indicating: “Help me plan my itinerary for my firsttrip to Japan.” This statement provided by the user is used to form thedirective of Command Card B (e.g. referenced as CC.B in the short form).

Regarding Command Card A 601, it includes: an ID data tag 602 whichuniquely identifies the command card; a directive module 603 that isconfigured to process user input to identify a directive of the commandcard 601; a memory module 604, which stores data related to the commandcard 601; a UI module 605 that includes executable instructionsoutputting information via one or more user devices, and for receivinguser input via one or more user devices; a notes module 606 forobtaining and processing data from a user (e.g. text data, oral data,image data, video data etc.) that is related to the command card 601;one or more search bots 607 for searching for data specific to thedirective; one or more behavior bots 608 for monitoring user behaviorand for processing the data obtained by the search bots; and a privacyand security module 609 for determining privacy settings related to dataor computing functions, or both. For example, a user can keep private orshare one or more of the following: resulting data of the command card A601, the data of the notes module 606, one or more of the search botsthemselves, and one or more of the behavior bots themselves.

The search bots of 607 for example, include: Search Bot 1-CC.A andnumerous other search bots including Search Bot n-CC.A. Each of thesesearch bots search a different data source to find and processinformation specific to the directive. It is herein recognized thatdifferent types of data require different types of computationalprocessing. For example, photo data, audio data, video data, text data,and machine data, each require different types of processing. Thebehavior bots 608, or example, include: Behavior Bot 1-CC.A and numerousother behavior bots, including Behavior Bot n-CC.A. Each behavior botmonitors a different behavior of a user, and processes the dataaccordingly. For example, one behavior bot monitors facial expressionsof a user to determine behavior, while a different behavior bot monitorsrisk tolerance patterns of the user, and while yet another behavior botmonitors travel patterns of the user.

In an example aspect, the user provides user input via the UI module tomodify the directive. In an example embodiment, the detected behaviorsof the user by one or more behavior bots, via the UI module, are used tomodify the understanding of the directive. For example, the directivemodule uses data science and machine learning to initially interpret auser's command or query. Over time, as the user interacts with the data,the directive module obtains data from one or more behavior bots tomodify one or more parameters in the data science computation or themachine learning computations, or both, to output a secondinterpretation of the user's command or query. This secondinterpretation then becomes the new directive of the command card.

The Command Card B (CC.B) 610 has similar data components, modules andbots. However, the directive, the search bots, and the behavior bots,amongst other things are different than those of Command Card A.

It can be appreciated that a user can form many different command cards.

In an example aspect, each command card is a digital container that isspecific to a user-defined given topic, or theme, or concept, or query.It contains machine intelligence to search for and provide personallyrelevant data for the given topic, or concept, or query.

In an example aspect, this digital container is private, personal, andsecure. In another aspect, this digital container has credentialcapability for the command car itself, for various levels of users, andfor the bots. This enables individuals to have private containers andanswers that only the user can see. This enables groups of people tocollaborate and keep the container content and answer private only tothat group. This enables employees or people associated with a businessor organization to collaborate and keep container content and answersprivate to that business or organization. This enables varying levels ofindividual, group, business, or organizations to pick and use onlycertain global bots, local bots, and user information.

Turning to FIG. 7, an example flow diagram of data between the differentcomponents of a command card is shown. The flows between the componentsor modules are marked with reference letters. In the example shown inFIG. 7, the given Command Card A has been created, for example, based oncertain user input.

At operation A, the directive module 603 sends the directive to each andevery one of the search bots 607. For example, this can occur whenduring the provisioning process of each search bot. The search bots thenexecute their search function of different data sources. Each search botsearches a different source to look for data that is relevant to thedirective provided by the directive module.

It can be appreciated that different search bots can use different dataprotocols to search data. For example, one search bot obtains data usinga push protocol, where a given data source pushes the data to the searchbot. In another example, a different search bot uses a pull protocol,such as for a web crawler bot. Furthermore, each search bot can begoverned by different internal policies. For example, one search botsearches a data source once every 24 hours, while a different search botsearches a different data source once every second. There may be otherdifferences between the search bots. The search bots operate in parallelto each other and repeatedly.

As per operation B, the results obtained by the search bots are temp ina memory module 604 associated with Command Card A. In operation C, thebehavior bots 608 filter or process the data obtained by the searchbots, or both, and store the filtered data or processed data (or both)in the memory module 604. This filtered data or processed data, or both,is relevant to the behavior of the user. In an example embodiment, thedata is filtered and then processed to obtain a summarization of thedata; this data along with the data links to the data source are storedin the memory module 604. In an example embodiment, the full or actualdata sources of the data links is accessible by the data links, but thefull or actual data sources are not stored in the memory module in orderto reduce the amount of memory resources (e.g. hardware and softwareresources).

In operation D, the resulting information is sent to the UI module 605,which then processes the data for outputting to one or more user devices701 via operation E. The user device or devices 701 could be the OCD301, or the user device 102, or some other user device or devices thatallow one or more people to interact with data. One or more usersinteract with one more user devices and, as part of operation E, the UImodule obtains user interaction data. These interaction data couldinclude, for example and without limitation, voice data, gesture data,text data, image data, facial data, audio data, biometrics data, brainsignal data, muscle signal data, nerve signal data, command data (e.g.clicks, swipes, dislike/like input, information regarding sharing of thedata, bookmarking of the data, highlighting of the data, etc.), or acombination thereof. The UI module 605 sends this detected userinteraction feature data to the behavior bots 608, at operation F.

The behavior bots 608 take this data as input to modify the parametersused in their filtering algorithms, or processing algorithms, or both.In addition or in alternative, a behavior bot may altogether changetheir filtering algorithm or processing algorithm, or both, based on theuser interaction feature data. This feedback in turn affects futureiterations of operation C to filter or process the data, or both, thatis obtained by the search bots.

For example, if a user shows an acceptance of uncertain information, orshows a willingness to act on information with higher degree ofuncertainty, then the user's risk tolerance is considered high. Thebehavior bot that is specific to risk tolerance then modifies its filterto accept data that has a higher degree of uncertainty, compared to itsinitial setting of filtering out data with a higher degree ofuncertainty.

In another example embodiment, in operation G, one or more of thebehavior bots also sends the determined behaviors of the user to the UImodule 605. The UI module 605, for example, uses these determinedbehaviors to modify the presentation of the data to be customized to thebehavior of the user.

In another example embodiment, one or more of the behavior bots alsosend(s) the determined behaviors of the user to the search bots inoperation H. The search bots 608, for example, use these determinedbehaviors to modify parameters of the searching and initial processingof the data to reflect the determined behavior of the user.

FIGS. 8 and 9 show example computing architectures of the dataenablement platform, which can be incorporated into the above computingsystems.

Turning to FIG. 8, an example computing architecture is provided forcollecting data and performing machine learning on the same. Thisarchitecture, for example, is utilized in the AI platform 107 and thedata science servers 104.

The architecture in FIG. 8 includes multiple data sources. For example,data sources include those that considered part of any one or more of:the IoT data sources, the enterprise data sources, the user generateddata sources, and the public data sources (e.g. public websites and datanetworks). Non-limiting examples of IoT devices include sensors used todetermine the status of products (e.g. quantity of product, currentstate of product, location of product, etc.). IoT devices can also beused to determine the status of users (e.g. wearable devices). IoTdevices can also be used to determine the state of users (e.g. wearabledevices), the environment of the user, or sensors that collect dataregarding a specific topic. For example, if a person is interested inweather, then IoT sensors could be weather sensors positioned around theworld. If a person is interested in smart cities, then IoT sensors couldinclude traffic sensors. Enterprise software can include CustomerRelations Management software, Enterprise Resources Planning Software,supply management software, etc. User generated data includes socialdata networks, messaging applications, email data, text data, blogs,user generated video data, audio blog data, and online forums. Publicwebsites and data networks include government websites and databases,banking organization websites and databases, economic and financialaffairs websites and databases. It can be appreciated that other digitaldata sources may be collected by the search bots. In an exampleembodiment, one or more the data sources are internal or private to agiven person or to a given organization. For example, enterprise datacan be internal to a given company and messaging data (e.g. viaFaceBook, MSN Messenger, WeChat, WhatsApp, SnapChat, etc.) can beprivate to a given person.

In particular, each one of the search bots in the data collectors module802 collect data from a specific data source. For example, Search Bot 1for Command Card A (SB1-CC.A) searches a first data source to obtaindata for the directive in Command Card A, and Search Bot 2 for CommandCard A (SB2-CC.A) searches a second data source to obtain data for thesame directive in Command Card A. Search Bot 1 for Command Card B(SB1-CC.B) searches a third data source to obtain data for the directivein Command Card B, and Search Bot 2 for Command Card B (SB2-CC.B)searches a fourth data source to obtain data for the same directive inCommand Card B.

The first, second, third and fourth data sources are, for example, alldifferent from each other.

In another example, the first and the second data sources are differentfrom each other, and the third and the fourth data sources are differentfrom each other. However, the first data source and the third datasource are the same. In other words, two different command cards canhave their respective different search bots that search the same datasource, but for different directives.

The search bots operate in parallel to generate parallel streams orthreads of collected data. The collected data is transmitted via amessage bus 803 to a distributed streaming analytics engine 804, whichapplies various data transforms and machine learning algorithms. Forexample, for Search Bot 1 for Command Card A (SB1-CC.A), the streaminganalytics engine 804 has modules to transform the incoming data, applylanguage detection, add custom tags to the incoming data, detect trends,and extract objects and meaning from images and video. It will beappreciated that other modules may be incorporated into the SB1-CC.A.Other search bots can have the same streaming analytics modules, ordifferent ones. For example, the streamlining analytics modules ofSearch Bot 2 for Command Card B (SB2-CC.B) includes generating text datafrom audio data (e.g. a pod cast or video) using natural languageprocessing and speech-to-text processing, identifying sentiment of thedata, and formatting the text data and the sentiment data. It can beappreciated that different data sources can use different reformattingprotocols. Each search bot processes its data using streaming analyticsin parallel to the other search bots. This continued parallelizedprocessing by the search bots allows for the data enablement platform toprocess large amounts of data from different data sources in realtime,or near realtime.

In an example implementation, the engine 804 is structured using one ormore of the following big data computing approaches: NiFi, Spark andTensorFlow.

NiFi automates and manages the flow of data between systems. Moreparticularly, it is a real-time integrated data logistics platform thatmanages the flow of data from any source to any location. NiFi is datasource agnostic and supports different and distributes sources ofdifferent formats, schemas, protocols, speeds and sizes. In an exampleimplementation, NiFi operates within a Java Virtual Machine architectureand includes a flow controller, NiFi extensions, a content repository, aflowfile repository, and a provenance repository.

Spark, also called Apache Spark, is a cluster computing framework forbig data. One of the features of Spark is Spark Streaming, whichperforms streaming analytics. It ingests data in mini batches andperforms resilient distributed dataset (RDD) transformations on thesemini batches of data.

TensorFlow is software library for machine intelligence developed byGoogle. It uses neural networks which operate on multiple centralprocessing units (CPUs), GPUs and tensor processing units (TPUs).

Analytics and machine learning modules 810 are also provided to ingestlarger volumes of data that have been gathered over a longer period oftime (e.g. from the data lake 807). In particular, behavior bots obtainuser interaction data to set parameters for filtering or processingalgorithms, or to altogether select filtering or processing algorithmsfrom an algorithms library. The behavior bots, for example, use one ormore of the following data science module to extract behaviors from theuser interaction data: an inference module, a sessionization module, amodeling module, a data mining module, and a deep learning module. Thesemodules can also, for example, be implemented by NiFi, Spark orTensorFlow, or combinations thereof. Unlike these the modules in thestreaming analytics engine 804, the analysis done by the modules 810 isnot streaming. The behavior bots then filter or process the data, orboth, that is outputted from the search bots in the engine 804. In anexample embodiment, the behavior bots, not only filter the data, butalso process the data to provide understanding and prediction inrelation to the directive of a given command card. In other words, theresults could include the filtered data, a summary of the filtered data,data links to the original data sources, understanding data (e.g. text,images, video, audio data, etc.) showing how the filtered data isapplicable to directive of the given command card, and prediction datathat provides predicted events or circumstances that are relevant to thedirective of the given command card. These results outputted by thebehavior bots are stored in memory (e.g. cache services 811), which arethen transmitted to the streaming analytics engine 604.

The results outputted by the streaming analytics engine 804, aretransmitted to ingestors 606, via the message bus 805. The outputteddata from the analytics and machine learning modules 810 are alsotransmitted to the ingestors 806 via the message bus 805.

The ingestors 806 organize and store the data into the data lake 807,which comprise massive database frameworks. Non-limiting examples ofthese database frameworks include Hadoop, HBase, Kudu, Giraph, MongoDB,Parquet and MySQL. The data outputted from the ingestors 806 may also beinputted into a search platform 808. A non-limiting example of thesearch platform 808 is the Solr search platform built on Apache Lucene.The Solr search platform, for example, provides distributed indexing,load balanced querying, and automated failover and recovery.

The memory modules of the search bots operate are provisioned and storedin the data lake 807 and the Solr search platform 808. For example, allthe results data stored in relation to SB1-CC.A is stored in a memorymodule that is stored in the data lake or is indexed using the Solrsearch platform, or both. Using this architecture, data relating todifferent search bots can be obtained from the data lake or the Solrsearch platform in parallel.

Data from the data lake and the search engine are accessible by APIservices 809.

Turning to FIG. 9, a continuation of the computing architecture fromFIG. 8 is shown, which is used after the data has been stored in thedata lake 807 and indexed into the search platform 808.

A core services module 702 obtains data from the search platform 608 andthe data lake 607 and applies data science and machine learningservices, distributed processing services, data persistence services tothe obtained data. For example, the data science and machine learningservices are implemented using one or more of the followingtechnologies: NiFi, Spark, TensorFlow, Cloud Vision, Caffe, Kaldi, andVisage. It will be appreciated that other currently known and futureknown data science or machine learning platforms can be used to executealgorithms to process the data. Non-limiting examples of distributedprocessing services include NiFi and Spark.

The API layer 809 includes various APIs that interact with theapplications 901, including the data enablement applications. The APIs,for example, exchange data with the applications in one or more of thefollowing protocols: HTTP, Web Socket, Notification, and JSON. It willbe appreciated that other currently known or future known data protocolscan be used.

The data enablement applications include various command cards (e.g.Command Card A and Command Card B) that each have their own directive. Ateam command card is also provided, which is an application that managesand organizes data from various command cards. The AI XD platform 109includes, for example, a plurality of intelligent devices, intelligentdevice message buses, and networks.

In an example aspect, the command card could be an application itself,or a widget that plugs into another application (e.g. Excel, a CRM, anERP, etc.).

In another aspect, the module 810 is shown in communication with theAPIs, but in another example, the module 810 can directly access thedata lake and solr search indexer.

Turning to FIGS. 10A and 10B, examples of team command cards aeprovided.

In FIG. 10A, the team command card is associated with a command card foruser and a command card for user 2. The team command card and theseassociated command cards all have the same directive module A. However,other attributes about the command cards may be different, such as thesearch bots and the behavior bots assigned to each of the command cards.The different data obtained by the command cards for the users are sentto the team command card. The team command card uses its unificationmodule to combine the data from the command cards for the users, whichincludes identify and deleting duplicate data. In other words, usingthis architecture, different command cards are used to obtain data fromdifferent sources, and being personalized to different users. Thesedifferent data with different personalizations are combined with theteam command card.

In FIG. 10B, the team command card is associated with different commandcards for user 1 and user 2. The team command card includes a directivemodule, a unification module to unify data from the different commandcards, and a distribution module. The distribution module includesexecutable instructions and memory for dividing the directive intosub-directives, and assigning different sub-directives to differentcommand cards. For example, the command card for user 1 has asub-directive module A and the command card for user 2 has asub-directive module B. In other words, different command cards obtainand process data to answer different questions or achieve differentgoals, which are part of an overall directive. For example, the overalldirective of the directive module in the team command card is to help afamily of users to plan an itinerary for their first trip to Japan. Thesub-directive module A has the directive to plan travel andaccommodations, and the sub-directive module B has the directive to planactivities and sight-seeing.

It is herein recognized that the devices, systems and the methodsdescribed herein enable the provision of relevant digital contentspecific to the interest of a given user.

In an example aspect, the devices in combination with the dataenablement platform provides people with “Perfect Information”, aconcept from economists.

The data enable platform described herein, in combination with the userdevice or the OCD, or both, provide perfect information to help a personconsume digital media content and to interact with the digital mediacontent. A user, for example, talks with a chatbot on the user device orthe OCD.

In preferred embodiments, the chatbot has language capabilities tointeract with the user via text language or spoken language or both.However, in other example embodiment, the command card user interfacedoes not necessarily chat with a user, but still affects the display ofdata being presented to the user.

The systems described herein provide a collection of command cards withintelligent bots tied to each command card. Each command card is createdor customized by a user and that represents a topic, theme, interest,query, research project, goal, etc. For example, a user can orally speakinto the application and say, “Hey Bot, create a black hole entanglementcommand card.” The application, in turn, creates a command card, selectsa picture from the web depicting black hole entanglement, and displayswords below the picture stating “Black hole entanglement.” In analternative example embodiment, the user uses their brain signals, nervesignals, or muscle signals, or a combination thereof, to generate userinput data that is converted to speech by the user device or the dataenablement program, in which the processed speech is ““Hey Bot, create ablack hole entanglement command card.”

It will be appreciated that the term “command card” herein refers to aunified collection of data that answers or is relevant to the directiveassociated with it. The data includes, for example, one or more of textdata, audio data and visual data (e.g. images or video, or both).

Various search bots associated with the command card for black holeentanglement begin autonomously searching the Internet news, blogs,forums, periodicals, magazines, social sites, video sites, etc.multimedia (text, audio, video, pictures) that closely match the keywords and phrase “black hole entanglement”. These search bots uses datascience, such as, but not limited to, K Means clustering, to identifyattributes and characteristics that most closely reflect the attributesand characteristics of black hole entanglement. The behavior botsassociated with this command card filter or process the found data, orboth.

The user subsequently selects the black hole entanglement command card,via the user interface, and consequently the command card beginsdisplaying summary information, pictures, article, videos, etc. ofinformation specific to black hole entanglement based upon the resultsafter filtering or processing, or both, by the behavior bots.

The user can orally or manually say in relation to each multimediapicture, text, audio, video, whether he or she likes or dislikes thecontent. A behavior bot begins learning what the user likes and dislikesabout the K Means results and subsequently tunes the data science topresent results that are more like the machine learned user “likes”.

The user, via the notes module of the command card, can also comment onthe content by via a microphone or via other sensors like brain signalsensors, muscle signal sensors or nerve signal sensors, or a combinationthereof. For example, the user may comment: This theory sounds familiar;or The new satellite from ABC Inc. should provide more facts thatsupport this theory. The command card, via the data enablement platform,uses this information to provide related information in the same digitalmagazine.

In a particular example, as the user reads, listens, or watches amultimedia piece, the user can tell the application to pause. At thispause point, the user can create voice and typed-in bot notes linked toa key words, phrases, pictures, video frames, sound bytes in themultimedia; this implemented using a pause point bot. These user-createdbot notes enable the user to insert thoughts, comments, reminders, todo's etc. and are indexed for future access. At this pause point, in analternative embodiment, the user can perform a search using searchengines such as Google or Bing. If the user likes one of the resultsfrom the search results page, the user can orally connect that linkedmultimedia to the digital magazine pause point for future reference. Atthis pause point, in an alternative embodiment, the user can orally linkto a different web site, forum, blog, forum, etc., search for results,and link this resulting information back to the pause point. The pausepoint bot can simultaneously begin searching for other Internetmultimedia documents, apply K means to the results and recommend othermultimedia documents that are very similar to each comment, to do,reminder, search result link, forum, blogs, news, periodical, etc—thisis akin to people who saw these results for a topic also searched andfound X multimedia, with has characteristics and attributes closelyrelated to a specific thought, to do, video, etc.

As the user reads, listens, and adds more related comments, notes,links, etc, to the black hole entanglement command card, the user hasthe option to publish and share his command card with other people viasocial media, forums, blogs, etc.

The user-created or user-appended notes associated with the command cardcan include documents, captured pictures/videos, audio recordings, andinputted IoT data.

As the user adds comments to the command card (e.g. via oral, typing,brain signals, nerve signals, muscle signals, etc.), a behavior botapplies sentiment analysis to the user-inputted comments creating metadata that can help the behavior bot understand excitement, sadness, etc.toward a piece (e.g. an article, a video, a blog entry, an audio cast orpodcast, etc.) in the command card.

As the user adds speech, picture, and video comments to the digitalmagazine, another behavior bot can record/observe for background noise,background picture/video elements (location, color, people, objects)creating meta data that can help the behavior bot to better understandcontext or the environment of where a user is consuming informationabout the black hole entanglement command card. For example, the dataenablement platform determines if the user is consuming the media whileon a train, or on a plane, or in a bathroom, or at a park, or withpeople around them, etc.

The user interface module of the black hole entanglement card can alsocompute and generate a visual graph data representation showing how allof the black hole entanglement media pieces are related to one anotherfor easy future access as well as propose and recommend other mediaarticles, web site, news, blogs, and forums to view and potentially addto the black hole entanglement digital magazine.

The data enablement platform also enables other people to follow auser's specific command card if the command card creator publishes andallows people to follow this command card.

In an example aspect, a person who has created a command card for acertain topic can adjust settings that direct the data enablementplatform to privately share the given command card with selectedcontacts, or to be shared publicly.

The system enables the command card creator to receive comments,questions, links, digital media and to decide whether to add thissubmitted information to the existing black hole entanglement commandcard.

In an example aspect, the results of the aformentioned information on aspecific topic, theme, interest, etc. results in the closest, real time,perfect information.

Based on these technical features, in effect to the user, a user who isan enthusiast no longer has to do searches that are deep and relevant tothe directive. The data enablement platform and the user device ordevices pull the information together for the user in an easy-to-consumeand interactive format.

Turning to FIG. 11, an example embodiment is provided of softwaremodules that reside on a given user device 1101, data science servers1102, and internal applications and databases 1103.

For example, a data enablement application 1104 resides on the userdevice and the application includes: a Command Card A, Command B,Command Card N, and so forth, an Team Command Card, and a configurationmodule. The user device also includes user interface (UI) modules 1105,which can be part of the data enablement application 1104, or mayinteract with the data enablement application 1104. The UI modulesincludes user interface bots (e.g. chatbots) that are associated with,or part of, each command card. For example, UI Bot A is linked toCommand Card A, and UI Bot B is linked to Command Card B. There is alsoa global chatbot that interfaces with the overall application 1104 andwith the command cards. The UI modules also include one or more GUIs,one or more messaging applications, one or more synthesizer voicemodules, and one or more tactile feedback modules, or combinationsthereof.

The data science servers 1102 include a data science algorithms library,a search bot library containing various search bots, a behavior botlibrary containing various behavior bots, a configuration module, and apolicy and rules engine. For example, the policy and rules engineincludes policies and rules that are specific to the company ororganization using the data enablement platform.

Regarding the data science algorithms library, it will be appreciatedthat data science herein refers to math and science applied to data inthe form including but not limited to algorithms, machine learning,artificial science, neutral networks, etc. The results from data scienceinclude, but are not limited to, business and technical trends,recommendations, actions, trends, etc. The search bots and behavior botsobtain executable code for conducting data science and machine learningfrom the data science algorithms library.

In an example aspect, Surface, Trend, Recommend, Infer, Predict andAction (STRIPA) algorithms are included in the data science algorithmslibrary. This family of STRIPA algorithms worth together and are used toclassify specific types of data science to related classes.

Non-limiting examples of other data science algorithms that are in thedata science library include: Word2vec Representation Learning;Sentiment (e.g. multi-modal, aspect, contextual, etc.); Negation cue,scope detection; Topic classification; TF-IDF Feature Vector; EntityExtraction; Document summary; Pagerank; Modularity; Induced subgraph;Bi-graph propagation; Label propagation for inference; Breadth FirstSearch; Eigen-centrality, in/out-degree; Monte Carlo Markov Chain (MCMC)simulation on GPU; Deep Learning with region based convolutional neuralnetworks (R-CNN); Torch, Caffe, Torch on GPU; Logo detection; ImageNet,GoogleNet object detection; SIFT, SegNet Regions of interest; SequenceLearning for combined NLP & Image; K-means, Hierarchical Clustering;Decision Trees; Linear, Logistic regression; Affinity Association rules;Naive Bayes; Support Vector Machine (SVM); Trend time series; Burstanomaly detection; KNN classifier; Language Detection; Surfacecontextual Sentiment, Trend, Recommendation; Emerging Trends; WhatsUnique Finder; Real-time event Trends; Trend Insights; Related QuerySuggestions; Entity Relationship Graph of Users, products, brands,companies; Entity Inference: Geo, Age, Gender, Demog, etc.; Topicclassification; Aspect based NLP (Word2Vec, NLP query, etc.); Analyticsand reporting; Video & audio recognition; Intent prediction; Optimalpath to result; Attribution based optimization; Search and finding; andNetwork based optimization.

In other example embodiments, the aforementioned data science can resideon the user's smartphone, or in public or private clouds, or at acompany's data center, or any combination of the aforementioned.

Continuing with FIG. 11, UI modules 1106 also reside on the data scienceservers 1102.

In an example embodiment, internal applications and database 1103 areused to assist in fulfilling the directives of command cards. Examplesof such software and databases include: email software, calendarsoftware, contact list software, project management software, CRMsoftware, accounting software, and inventory software. Each of thesedifferent software are different data sources, and different search botsare used to respectively search these different software.

FIGS. 12 and 13 include screenshots of example GUIs shown for applyingthe data enablement system to the display data for a command card.

In FIG. 12, a home landing page 1201 is shown for the data enablementapplication. It includes a search field 1202 to receive text input fortopics, names, things, etc. A user can also speak to the global chatbotto explore or search for topics, names, things, etc. It also includesGUI controls 1203, 1203 for activating each command card. For example,the control 1203 represents a command card about black hole entanglementand the control 1204 represents a different command card about planningan itinerary for a first trip to Japan. By receiving a selection (e.g.either through a GUI or by an oral command) of one of these controls,the user device will launch a GUI specific to the selected command cardand will activate the corresponding UI bot.

FIG. 13 shows an example GUI 1301 of a selected command card. The layoutand format of the content can change over time, and can vary from userto user. The GUI can include text, video, or images, or a combinationthereof. A text field 1302 receives text input to initiate searches orto store comments related to a given digital media piece. The display ofvisual content can be scrolled up or down, or can be presented as pagesthat can be turned over.

By selecting a piece of content in the GUI, the UI bot begins to readaloud the content.

It is appreciated that the content in the command card user interfacecan be updated in realtime, even while the user is viewing the GUI, ascontent is procured by the search bots and the behavior bots of thecommand card.

The depicted control elements are for example. Other control elementswith different data science, bots, features, and functionality may beadded and mixed with other control elements.

Below are example questions and statement posed by a user, and oralfeedback provided by the UI bot (e.g. chatbot). It will be appreciatedthat the UI bot is conversational and adapts to the style of the user towhich it is speaking.

Example 1

User: Hey Bot, provide me with articles about topic X.

Bot: Hey User, here are the most recent articles about topic X and themost cited articles about topic X.

The Bot reads out summaries of the latest 3 new articles pulled fromvarious data sources, and reads out summaries of the 3 most citedarticles.

Example 2

User: Hey Bot, read article XYZ for me.

Bot reads out the article XYZ.

User: Hey Bot, please repeat the last few sentences.

Bot re-reads the last three sentences, pauses, and then continuesreading the rest of article XYZ.

Example 3

User: Hey Bot, read article XYZ for me.

Bot reads out the article XYZ.

User: Hey Bot, I think the perspective on theory R is interesting.Professor P is doing some research to disprove it.

Bot: Hey User, I have found more content about theory R, articles fromProfessor P about theory R, and other content about disproving theory R.Do you want to hear this content now or save it for later?

User: Hey Bot, continue reading the article and then read me thearticles from Professor P.

Bot continues to read out the article XYZ. Afterwards, Bot reads out thearticles from Professor P.

Example 4

User: Hey Bot, show me the current production capacity.

Search bots gather IoT data regarding manufacturing machineryproductivity and the command card UI bot generates a graph showingproduction capacity of each piece of machinery.

Turning to FIG. 14, example computing operations are implemented by thedata enablement platform to search for and output personalized data to auser.

Block 1401: Based on user input, the data enablement platform creates acommand card. This includes the directive module receiving auser-inputted directive, and processing the same to generate a computerreadable directive.

Block 1402: The data enablement platform assigns and provisions searchbots to the command card. The provisioning includes creating each searchbot that searches a different data source to fulfill the directive. Thiscan be done automatically or semi-automatically. For example, a user canselect, via a UI, the data sources to be searched, which in turndetermines the type of search bots. The user can also adjust parametersof the assigned search bots, thereby customizing the search bots. Forexample, the user can input certain keywords, names, or types of data,for a given search bot to use in their searching computations.

Block 1403: The data enablement platform assigns and provisions behaviorbots to the command card. This includes creating each behavior bot tomonitor, understand, and potentially predict behavior data, in thecontext of the command card and its directive. This can be doneautomatically or semi-automatically. For example, a user can selectcertain behaviors to be monitored. The user can also customize thebehavior bots to have certain biases that are specific to a givencommand card.

Block 1404: The data enablement platform provisions storage space forthe command card. For example, the storage space is provisioned in thedata lake.

Block 1405: The data enablement platform assigns privacy/securitysettings to search bots, behavior bots, or storage space, or acombination thereof. This can be done according to a default setting,and can be customized by the user.

Block 1406: The search bots are executed. Although the search bots havethe same directive, they operate in independently and in parallel toeach other.

Block 1407: The data enablement platform caches, indexes and storesresults obtained by the search bots.

Block 1408: The behavior bots filter and process the results obtained bythe search bots to generate personalized results.

Block 1409: As part of the processing by the behavior bots, or by theoverall command card, the data enablement platform formats thepersonalized results for output via the user interface.

Block 1410: The user interface module of the command card outputs theformatted personalized results via one or more user devices.

Block 1411: The data enablement platform detects user interaction viathe user interface module, which are responsive to outputted results.The types of user interaction depend on the types of user devices anduser interfaces provided in this system.

Block 1412: The behavior bots use the detected user interaction data asinput to affect the data science of the behavior bots. These revised orself-modified behavior bots are used in future iterations of filteringand processing data.

Block 1413: The search bots obtain data from the behavior bots to updatedata science algorithms used by the search bots. These revised orself-modified search bots are then used in future iterations forsearching for data. The process repeats at block 1406 and onwards.

In an example embodiment, the process is continuous from block 1406onwards, with the search bots continuously searching for data and eachproduce their own data stream.

Turning to FIG. 15, according to another example embodiment, acomputational flow diagram is shown for using the search bots and thebehavior bots. Different search bots 1501 are used to search differentdata sources. For example, there are different search bots correspondingto each of the different data sources: a search bot for social data, asearch bot for a given search engine, a search bot for a career website,a search bot for IoT data, a search bot for virtual or augmented realitydata, a search bot for edge device data, a search bot for enterpriseresource planning (ERP) data, a search bot of music data, a search botfor video data, and a search bot for 3^(rd) party data.

As per block 1502, each search bot performs search operations forrelevant data. For example, a crawler technology is used in some casesto obtain the relevant data. Each search bot ingests and caches the dataresults. Each search bot applies data science to the data results. Ifuser defined keywords, hashtags, queries are provided, then the searchbots apply this user-defined input to the data results. Each search botfilters, surfaces and generates a revise set of data using the aboveoperations, to generate Set A data set.

At block 1503, the data of Set A includes all the relevant answers,recommendations, and information that are relevant to the directive ofthe command card. This information is cached, indexed, and data links tothe source data are stored. The source data can include one or more ofthe following: pictures, videos, social content, music, IoT data, edgedevice data, 3^(rd) party data, ERP data, virtual reality or augmentedreality data, career site data, news, blogs, forums, and other Internetsites.

The behavior bots 1504 personalize the Set A data, which results in SetB data 1505. In an example embodiment, the behavior bots 1504 include amix of one or more local behavior bots and one or more global behaviorbots. For example, a local behavior bot is specific to the user. Aglobal behavior bot is specific to a group of users (e.g. anorganization) of which a given user is a member. For example, a givenlocal behavior bot is specific to interests or biases of a given user,and a given global behavior bot is specific to ethics that represent thegroup of users.

The Set B data is a subset of the Set A data, and it may be furthertransformed in its summarization and presentation to reflect thebehavior data features determined by the behavior bots.

At block 1505, the Set B data includes data that is tailored to a givenuser, a given group, a given department, a given organization, etc.based on the machine learned profile generated by the behavior bots.This Set B data is cached, indexed, and the links to the source data arestored. This Set B data may also be ranked and scored for relevancy.

At block 1506, the answers, recommendations, and information of the SetB data are presented via APIs, applications, and one or more userdevices.

The outputs of block 1505 are used to inform the search bots, which canadjust their search and data science parameters based on certainbehavior features.

Turning to FIG. 16, an example computation is shown for applying naturallanguage processing (NLP). At block 1601, the user device or the OCDreceives input to monitor a given topic. At block 1602, at regularintervals (e.g. daily), the data enablement platform executes externalsearches for the latest news regarding a given topic. At block 1603, theexternal search results are stored in memory. At block 1604, the dataenablement platform applies NLP automatic summarization of the searchresults and outputs the summarization to the user device (e.g. via audiofeedback) (block 1605). The process then repeats at regular intervals,as per block 1602.

Turning to FIG. 17, another example computation is provided. At block1701, the user device or the OCD receives input to obtain informationabout a given topic, as per a directive of a command card. At block1702, at regular intervals (e.g. daily), the data enablement platform,via search bots, executes external searches for the latest newsregarding a given topic. At block 1703, the external search results arestored in memory. At block 1704, the data enablement platform, viasearch bots, executes internal searches for the given topic. At block1705, these internal search results are stored. At block 1706, the dataenablement platform compares the external search results with theinternal search results to determine if they affect each other. Forexample, the data enablement platform determines if there aredifferences in the data or similarities in the data, or both. At block1707, the data enablement platform applies NLP automatic summarizationof the affected external search results, or the affected internal searchresults, or both. The summarization is outputted to the user device forvisual display or audio feedback (block 1708). In this way, a user isinformed of relevant news and why the news is relevant (e.g. affectedinternal data, etc.).

In an example embodiment, the above methods in FIG. 16 or 17 are used toprovide a UI bot, or chatbot, that provides a fast and easy way toconsume news summaries (e.g. news releases, investigative articles,documentaries, LinkedIn, Facebook fan page, etc.) for each specifictopic.

Turning to FIG. 18, example executable instructions are provided forusing dynamic searches to affect the way certain data is outputted atthe user device.

Block 1801: While the user device plays audio of text, the user devicedetects a user's oral command to at least one of: repeat a portion oftext, search a portion of text, clarify a portion of text, comment on aportion of text, highlight or memorialize a portion of text, etc.

Block 1802: The user device or the data enablement platform, or both,executes the user's command.

Block 1803: The data enablement platform globally tallies the number oftimes the certain portion of text is acted upon by any and all users, orcertain highly ranked users, or both.

Block 1804: After a certain number of times has been counted, the dataenablement platform tags the certain portion of text.

Block 1805: When the certain portion of text, which is tagged, is beingplayed via audio means via an ancillary user device, the user deviceplays the audio text with emphasis (e.g. slower, louder, in a differenttone, in a different voice, etc.). In other words, the data enablementplatform has tagged the certain portion of the text and has performed anaudio transformation on the certain portion of text.

Therefore, if User 1 comments on some text or audio or video, when User2 reviews the same data, the chatbot for User 2 will read out the textwith emphasis. In an example embodiment, User 2 does not know whatcomments are, but only that the portion of text was considered importantby many users.

Turning to FIG. 19, example executable instructions are provided forprocessing voice data and background noise.

Block 1901: The user device or the OCD records audio data, includingvoice data and background noise.

Block 1902: The data enablement platform applies audio processing toseparate voice data from background noise.

Block 1903: The data enablement platform saves the voice data and thebackground noise as separate files and in association with each other.

Block 1904: The data enablement platform applies machine learning toanalyze voice data for: text; meaning; emotion; culture; language;health state of user; etc.

Block 1905: The data enablement platform applies machine learning toanalyze background noise for: environment, current activity engaged byuser, etc.

Block 1906: The data enablement platform applies machine learning todetermine correlations between features extracted from voice data andfeatures extracted from background noise.

In this way, information about the user can be more accuratelydetermined, such as their behavior and their surroundings. Thisinformation is stored as part of a given user profile (e.g. User 1Profile, User 2 Profile, etc.). This in turn can be used curate morerelevant content to a user, identify similar users, format the output ofthe content (e.g. language, speed of reading, volume, visual layout,font, etc.) to meet the profile of the user, and provide data topublishers and content producers to generate more relevant content.

Turning to FIG. 20, example computer executable instructions areprovided for processing data using the data enablement platform. Atblock 2001, a user device or an OCD, or both, receives input to select afunction or a mode of an application (e.g. the data enablementapplication) that resides on the user device. At block 2002, the userdevice or the OCD, or both, obtains voice data from a user. At block2003, the user device or the OCD, or both, transmits the same data tothe 3^(rd) party cloud computing servers. The user device alsotransmits, for example, contextual data. At block 2004, the 3^(rd) partycloud computing servers processes the voice data to obtain datafeatures.

Non-limiting examples of extracted data features include text,sentiment, action tags (e.g. commands, requests, questions, urgency,etc.), voice features, etc. Non-limiting examples of contextual featuresinclude the user information, device information, location, function ormode of the data enablement application (e.g. the selected commandcard), and a date and time tag.

The extracted data features and the contextual features are transmittedto the data science servers (block 2005). The original data (e.g. rawaudio data) may also be transmitted to the data science servers. Atblock 2006, the data science servers process this received data.

At block 2007, the data science servers interact with the AI platform,or the internal applications and internal databases, or both, togenerate one or more outputs.

The data science servers then send the one or more outputs to the 3^(rd)party cloud computing servers (block 2008). In one example embodiment,the 3^(rd) party cloud computing servers post-processes the outputs toprovide or compile text, image, video or audio data, or combinationsthereof (block 2009). At block 2010, the 3^(rd) party cloud computingservers transmit the post-processed outputs to the relevant userdevice(s) or OCD(s). At block 2011, the user device(s) or the OCD(s), orboth, output the post-processed outputs, for example, via an audiodevice or a display device, or both.

In an alternative embodiment, stemming from block 2008, the 3^(rd) partycloud computing server transmits the outputs to the one or more relevantdevices (e.g. user devices or OCDs) at block 2012. The post-processingis then executed locally on the one or more relevant devices (block2013). These post-processed outputs are then outputted via audio devicesor visual devices, or both on the one or more user devices or OCDs(block 2011).

Turning back to block 2007, in an example aspect, the data scienceservers pull data from the internal applications and internal databases,or the internal applications and internal database are updated based onthe results produced by the data science servers, or both (block 2014).

In another example aspect, the data science servers transmit data andcommands to the AI platform, to apply AI processes on the transmitteddata. In return, the AI platform transmits external and localinformation and data intelligence to the data science servers. Theseoperations are shown in block 2015.

It can be appreciated that any two or more of the operations in blocks2007, 2014, and 2015 can affect each other. In an example embodiment,the outputs of block 2014 are used in the operations of block 2015. Inanother example embodiment, the outputs of block 2015 are used in theoperations of block 2014.

Using the systems, devices and computing operations described herein,instead of wasting time trying to search and analyze all the latest andgreatest information, a user can merely speak, gesture, and input data,photos, videos, audio, keywords, hashtags, etc. into the related touser's defined command card. Each digital command card represents auser's interest, search, project, opportunity, etc. The search botsautonomously search, sort, and filter the search result data using datascience (AI, ML, STRIPA) and create a set of possible answers. As theset of answers becomes arriving, the system begins applying a differenttype of data science bots. These behavior bots are user behavior andprofile bots that machine learn and understand my interests, likes,dislikes, etc. These behavior bots are applied to the set of possibleanswers and ultimately surface answers that are tailored to my behaviorand profile. For example, using K value across the returned searchresult answers and my machine learned behavior profile, only the topanswers or results would be prioritized and displayed in rank order.There could be hundreds of potentially right answers but only 2 or 3that match my personality, behavior, or profile. Providing these 2 or 3answers saves invaluable time to the user and facilitates making themost informed, “perfect information” decision that is available for thatperson.

Furthermore, the ability to creatively input and actively engage withthe command card multimedia and data creates even more creative andricher answers using interactive devices such as augmented reality andvirtual reality devices and rooms and wearable devices.

Example Process of Creating and Using a Command Card

At a first step, a user creates a digital command card and assigns nnumber of search bots to the command card. The user can type in or speaka command card name. In an example embodiment, the user selects existingbots or creates user defined bots.

Non-limiting example of search bots, which are specific to a given datasource, include: Google bot; Bing bot; LinkedIn bot; FaceBook bot; news,blogs and forums bot; Fitbit bot; engine low oil IoT sensor bot; trainbraking system IoT sensor bot; network monitoring edge device; ERP bot;music bot; video bot; user defined query bot; augmented reality andvirtual reality wearables and room bot; and 3rd party proprietary querybot.

In another aspect of the first step, the user can create their ownunique bots. For example, a user can create a user defined query bot.

In another aspect of the first step, the user can reuse previouslycreated bots.

In another aspect of the first step, the search bots retrieve dataincluding pictures, videos, logs, text, machine data, and audio. Thesearch bots perform data science against the pictures, videos, logs,text, machine data, and audio data. Non-limiting examples of datascience processes include: k stat for grouping, and nearest neighbor forgrouping similar cohorts.

In another aspect of the first step, behavior bots are provisioned aspart of the command card. Baseline behavior are machine learned botsthat apply overarching machine learnings to all command card searchresults. Each of these behavior bot monitors a different behaviorfeature that is specific to a given person or group, and uses the samefor processing the data found by the search bots. Non-limiting examplesof behavior bots include: my behavior pattern bot; my personalitypattern bot; my spending pattern bot; my risk tolerance pattern bot; myaffinities pattern bot; my interest pattern bot; my health pattern bot;my facial expression pattern bot; my spoken and tone pattern bot; mygesturing pattern bot; my sentiment pattern bot; my demographic patternbot; my leisure bot; my relationship bot; my travel pattern bot; myeconomic status pattern bot; my education status bot; my sexualpreferences pattern bot; my mentorship bot; my family bot; my work bot;my friends bot; my ethics bots; my learning bot; my bio-signal bot (e.g.for detecting one or more of brain signals, muscle signals and nervesignals); my optimism bot; and my Myers Briggs-like personality traitpattern bot.

These behavior bots, for example, are assigned with baseline behaviorparameters that are known to a user. However, the behavior bots canevolve over time and become specific to a given command card. Forexample, a risk tolerance pattern bot for a first command card for auser, is different from a risk tolerance pattern bot for a secondcommand card for the same user. In another example, my machine learnedbehavior pattern is modified for a specific command card as a userbiases the local bot (example like/dislike content that is differentfrom my normal behavior pattern). Biasing the local bot examples includeliking and disliking command card results that a user reads and reactstowards. In another example, biasing the behavior bot also includesapplying machine learning to recognize that the user clicks on morearticles, pictures, videos relative to other articles, pictures, orvideos. In another example, biasing the behavior bot includes machinelearning oral notes (via NLP) of a user, or the tone of voice of theuser, or understanding the sentiment of these notes, or a combinationthereof. As a result, the behavior bot incorporates the user note biasinto the local bot for screen or rank ordering the answers.

At a second step, the user initiates and operationalizes the commandcard. It includes the following steps:

1) Orally communicate, type in, or gesture a command card creationcommand into the data enablement application.

2) The data enablement application orally or prompts the user for thename of the command card.

3) User names the command card.

4) The data enablement application prompts user to select bots (e.g.search bots and behavior bots) from a library.

5) The user picks and selects bots to the command card.

6) The data enablement application prompts user to add people(optionally) to CRUD (create, read, update, delete) information in thecommand card for collaboration purposes.

7) User picks and selects people to CRUD information in the commandcard.

8) The data enablement application saves the newly created command card.

9) The data enablement application begins executing the bots previouslyselected.

10) The data enablement application send invites to people if they wereadded to the digital container for collaboration

Below are the operations used to create user defined bots, as opposed toa user selecting from a library of existing bots.

1) Execute command to create bot.

2) Name the bot.

3) Define data source (social sites, enterprise database, lot/Edgedevices, news blogs forums, etc.).

4) Add keywords, hashtags, special queries/filters when performingsearch.

5) Select, parse and integrate data source API.

6) Select and integrate data science (ML, AI, STRIPA) that will beapplied to the data source search results.

7) Define technical and business rules and policies for searching andstreaming data. For example, some forums have crawling rules thatprevent a computing system from an IP address (internet protocoladdress) from crawling their site more than once a day. If that rule isbroken, the crawler bot is rejected from crawling on a go forward basis.

8) Save the newly created bot.

9) Add the newly created bot to the library of existing bots.

Below are example steps for picking existing bots from a library ofbots.

1) Execute opening a new or existing command card.

2) Open bot library.

3) Pick and choose n number of bots from the bot library.

4) For each bot, the user can open the bot and read the bot searchcriteria.

5) The user can add, update, and delete bot specific keywords, hashtags,queries,

6) The modified bot is saved in the library and also as part of thecommand card.

7) Save the command card with newly added bots.

The user can read, add, update, and delete bots within a command card atany time.

At a third step of the overall process, the command cards beginsexecuting the bots. This includes the following operations:

1) The search bot begins returning results.

2) Each command card caches the bot search results.

3) Data science is applied to each command card search bot results. Forexample, the data science includes determining relevance; determiningkey word/hashtag match; and determining the K stat; and nearestneighbor.

At a fourth step in the overall process, the results from the datascience (the second step) are cached, indexed and stored.

In an aspect of the fourth step, the bot index and store links tophotos, video, text, audio, machine data, IoT edge device, enterprisedata, and other data. The bots do not store all the content, only theanswers or a summary, and a link to the source data. The answers or thesummary includes a picture or a text summary as a reminder to the user.The data enablement system therefore saves resources by avoiding storingknown data. The data enablement system stores the links and pathways tothe data from disparate locations that results in an answer, which isrecognized to be more effective and efficient when extreme data isinvolved.

At a fifth step in the overall process, the behavior bots aresubsequently applied to the cached results obtained by the search bots,in order to tailor the set of “all possible answers” to a small “set ofcorrect answers that also match my behavioral profile”. In an examplefeature, the processed results are force ranked, scored, and displayed.In another example embodiment, although there may be potentiallyhundreds of right answers uncovered by the search bots, only 2 or 3answers are displayed based upon the machine learned bot behaviorprofile and that are the correct answers for “me” (e.g. a given user).In another example aspect, a score is applied to all the possibleanswers provided by the search bots, based on the behavior botsprocessing, which the behavior bots can use in future iterations formachine learning.

At a sixth step, the user can interact with the entire set of allpossible answers provided by the search bots, or just the 2 or 3 answersselected by and presented by the behavior bots in order to creativelyexplore for other interesting and related topics, ideas, places, people,events, etc. This information can be explored via a GUI that shows aVenn diagram of all possible answers and the selected 2 or 3personalized answers as a subset of the possible answers.

In an aspect of the sixth step, user interaction can include oralnavigated or type in creative exploration. In another aspect of thesixth step, user interaction can include augmented reality or virtualreality creative exploration. In another aspect of the sixth step, userinteraction can include wearable device creative exploration. In anotheraspect of the sixth step, user interaction can include voice notes andmeeting notes—annotations. In another aspect of the sixth step, userinteraction can include adding pictures, videos, text, audio to the“answers”. In another aspect of the sixth step, the results from any orall of the aforementioned creative exploration techniques can then beadded to bias or adjust the behavior bots. In another aspect of thesixth step, the results from any or all of the aforementioned creativeexploration techniques can add new content, hashtags, keywords, links,data to the existing command card. In another aspect of the sixth step,the user can elect to chain together other command cards that mightbecome related over time.

At a seventh step, the second step to the sixth step autonomously andcontinuously run in order to continuously provide the most up to date“perfect information” answer.

Example Aspects of the Search Bots

The data science driven search bots search or crawl for data, or both,and cache the data from data sources. The search bots also perform datascience on the cached search results, and filter and surface relevantresults,

The filtered and surfaced results is the Venn diagram of information,data, pictures, videos, data logs, machine data, IoT and edge devicedata, VR/AR data etc. that is relevant to the command card and searchbot criteria. One or more search bots can be assigned to a command card.

In an aspect, the search bot collects information from a single datasource or multiple data sources.

In another aspect, the search bot follows rules and policies (e.g.number of times a bot can crawl/query a site per day).

In another aspect, the results of the search bot are cached. Forexample, caching can occur at the edge device, smart phone device, IoTdevice, or in a public or private cloud, or any combination of theaforementioned.

In another aspect, the search bot nonstop executes the analysis,including applying data science against the cached search results thatare streaming in. The data science filters and surfaces only data,information, and answers related to the command card directive and thebot criteria. For example, the search bot criteria includes filteringand surfacing the user entered (optional) matching keywords, hashtags,etc. applied to the cached search results.

The output from performing the analysis becomes the Set A data that ismost relevant to the command card directive. The output results arepresented in form of information, answers to a question, pictures, data,videos, machine data, machine data, audio data, etc.

In another aspect, the Set A data is temporal and, therefore can becomeoutdated as new streaming data is provided by the search bots. Forexample, each bot continuously ingests streaming data, which in turncould modify Set A to Set A prime at time T+1, Set B double prime atT+2, etc. In another example, each bot could automatically search andcrawl data sources on an hourly basis, daily basis, weekly basis,monthly basis.

Example Aspects of the Behavior Bots

Data science driven behavior bots are behavior, personality,personality, profile bots that use behavioral data for training themachine learning algorithms.

These behavior bots can be for a specific person, a group of people, adepartment of people, a business, an organization, a one-time event ofpeople, a group of people who have similar interests, affinities,preferences, or inclinations, an object, an avatar, a place, an event,an action, a time period, a theme, a period or time window in a personor object's, a historical period in time, a future period in time, orfor a combination of these characteristics.

The behavior bots apply their data science against the “filtered andsurfaced” result set obtained from the search bots, also called Set Adata. The output from the behavior bots is information and answers thatare tailored to the person's behavior, profile, etc. or a group ofpeople's behavior, profile, or a department of people's behaviorprofile, a business behavior, profile, etc. and so forth as listedabove. In an example aspect, the command card provides highly tailoredrecommendations, answers, and information that are tailored to myindividual profile, my groups profile, department profile, etc., alsocalled Set B data. This small group of Set B data is different than theSet A data, which can have hundreds of correct recommendations, answersand information that are very good but are not suited to the profile andbehavior of myself or groups of like-minded people.

Like the Set A data, the Set B data is temporal.

The computing architecture described herein, and related systems anddevices, include multiple search bots and multiple behavior bots. Thesesearch bots operate in parallel, and each search bot provides a realtime feed dedicated to each data source. For each search bot, there isspecific data science and processing that is optimized for each datasource.

Similarly, the behavior bots operate in parallel, and each behavior botis dedicated to monitoring a certain behavior attribute of user. Eachbehavior bot can filter out incoming streams of data, in parallel, andincludes specific data science and processing optimized for eachbehavior bot. The behavior bots also advantageously collect behaviorattributes and transform and combine data.

The distinction and separation between the search bots and the behaviorbots allow the two classes of bots to operate in parallel to each other.Furthermore, the search bots and the behavior bots are specialized indifferent operations compared to each other. Furthermore, thedistinction and separate between the search bots and the behavior bots,and also amongst the search bots, and also amongst the behavior bots,facilitates transferability and customizability of each both. It alsoallows for the overall computing architecture to be massively scalableto a massive number of users for many different types of directives.

In an example embodiment, the user device, including and not limited tothe OCD, includes an onboard voice synthesizer module to generatesynthesized voices. Turning to FIG. 21, the onboard voice synthesizer isa Digital Signal Processing (DSP) based system that resides on the userdevice. It includes one or more synthesized voice libraries. It alsoincludes a text processor, an assembler, a linker module, a simulator, aloader, a DSP accelerator module which is managed by a hardwareresources manager, and a voice acquisition and synthesis module (e.g. ananalog/digital converter and digital/analog converter). The voiceacquisition and synthesis module is in data communication with amicrophone and an audio speaker.

FIG. 22 shows an example subset of components on a user device, whichincludes a DSP board/chip, an ADDA2 board/chip, a local bus of the DSPboard, a host bus, and a CPU of the smart device. These components, forexample, support the software architecture shown in FIG. 21.

It will be appreciated that different software and componentarchitectures (i.e. different from the example architectures shown inFIGS. 21 and 22) in a user device can be used to facilitate outputtingsynthesized voice data.

In an example embodiment, a given behavior bot detects the currentbehavior of the user and selects a synthesized voice library to generatea voice presentation of data to the user that is responsive to thecurrent behavior of the user. For example, if the user is detected to besad, the behavior bot selects a synthesized voice library that ischeerful. In addition or in alternative, the synthesized voice librarysynthesizes the voice of a person that is familiar to the user (e.g.their friend, their family member, their coach, their mentor, etc.). Inanother example, if the user is detected to be depressed or umotivated,the behavior bot responsively selects a synthesized voice library of acoach which has motivating voice characteristics, and generates a voicepresentation of data using the coach's synthesized voice library.

Additional general example embodiments and aspects are described below.

In a general example embodiment, a distributed computing system isprovided that includes server machines that form a data enablementplatform, the data enablement platform comprises: a plurality of datacollectors that stream data over a message bus to a streaming analyticsand machine learning engine; a data lake and a massive indexingrepository for respectively storing and indexing data; a behavioralanalytics and machine learning module; and multiple applicationprogramming interfaces (APIs) to interact with the data lake and themassive indexing repository, and to interact with multiple applications.The multiple applications comprise multiple command cards. A givencommand card is specific to a given user and the given command cardcomprises a directive module that stores a given directive, a memorymodule, one or more search bots that search for data that is relevant tothe given directive, and one or more behavior bots that process the dataobtained by the one or more search bots according to one or morebehavioral attributes of the given user.

In an example aspect, the one or more behavior bots apply one or moreartificial restrictions to the data obtained by the one or more searchbots, and the one or more artificial restrictions are associated withthe one or more behavioral attributes of the given user.

In another example aspect, the one or more behavior bots combine datafrom a different topic to the data obtained by the one or more searchbots, and the data from the different topic is associated with the oneor more behavioral attributes of the given user.

In another example aspect, the one or more search bots search for thedata that is relevant to the given directive taking into account one ormore criteria related to the given directive; and the one or morebehavior bots assess one or more attributes of the data obtained by thesearch bots; wherein the one or more attributes of the data areunrelated to the given directive and are associated with the one or morebehavioral attributes of the given user.

In another example aspect, the data obtained by the search bots isranked according the one or more attributes.

In another example aspect, the one or more behavior bots generate avoice presentation of the data obtained by the one or more search botsin a voice that is responsive to a current behavioral attribute of thegiven user.

In another example aspect, if the given user is detected to be in a sadmood or agitated mood, the one or more behavior bots generate the voicepresentation in a cheerful voice.

In another example aspect, if the given user is detected to be in a sadmood or agitated mood, the one or more behavior bots generate the voicepresentation in a voice that is familiar to the given user.

In another example aspect, if the given user is detected to be in a busyor concentrated mood, the one or more behavior bots generate the voicepresentation in a neutral voice.

In another example aspect, if the given user is detected to be in a busyor concentrated mood, the one or more behavior bots generate the voicepresentation with words that are spoken quickly.

In another example aspect, the computing system further comprises one ormore synthesized voice libraries, wherein each of the one or moresynthesized voice libraries comprise voice parameter features of one ormore corresponding people, and the one or more behavior bots select atleast one of the synthesized voice libraries to generate the voicepresentation.

In another example aspect, the voice parameter features comprise two ormore of: tone; frequency; loudness; rate at which a word or phrase issaid; phonetic pronunciation; lexicon; syntax; articulation; rhythm; andmelody.

In another example aspect, in the given command card, there are multiplesearch bots that are each assigned to a different data source, and themultiple search bots all search for data that is relevant to the givendirective of the given command card.

In another example aspect, the plurality of data collectors comprisemultiple search bots from the multiple command cards, and each searchbot generates a separate and parallel stream of data.

In another example aspect, the streaming analytics and machine learningengine comprises the multiple search bots from the multiple commandcards, and each search bots generates a parallel stream of data.

In another example aspect, a first search bot has a first set ofstreaming analytics modules to process data collected from a first datasource, and the second search bot has a second set of streaminganalytics modules to process data collected from a second data source.

In another example aspect, the behavioral analytics and machine learningmodule comprises multiple behavior bots that monitor user interactiondata via a user interface, and further process the data to obtainpersonalized data.

In another example aspect, the memory module of a given command card isprovisioned on the data lake and the massive indexing repository, andthe personalized data is stored in the memory module of the givencommand card.

In another example aspect, data links to source data and datasummarizations of the personalized data are stored in the memory moduleof the given command card.

In another example aspect, the given directive is obtained by processingspeech data of the given user.

In another example aspect, the speech data is derived from data recordedby a microphone that detects the given user's voice.

In another example aspect, the speech data is derived from data recordedby at least one of a brain signal sensor, a muscle signal sensor and anerve signal sensor that is on the given user.

In another example aspect, the one or more behavioral attributes of thegiven user are derived from user data obtained using at least one of abrain signal sensor, a muscle signal sensor, and a nerve signal sensorthat is on the given user.

In an example aspect, the oral computing device is a wearable device todynamically interact with the data. For example, the wearable deviceincludes inertial measurement sensors. In another example, the wearabledevice is a smart watch. In another example, the wearable device is aheadset. In another example, the wearable device projects images toprovide augmented reality.

In another example aspect, the oral computing device projects lightimages on surrounding surfaces to provide augmented reality of virtualreality. In another example aspect, the oral computing device is in dataconnection with other devices that projects light images to provideaugmented reality or virtual reality in a room. In effect, people thatare physically present in the room, or virtual people being displayed bythe projected light images, simultaneously interact and collaborate witheach other using their command cards.

In an example aspect, the oral computing device includes a graphicsprocessing unit (GPU) that exchanges data with the processor, the GPUconfigured to pre-process the audio data using parallel threadedcomputations to extract data features, and the data communication devicetransmits the extracted data features in association with the contextualdata and the audio data.

In an example embodiment, the oral computing device is a user device 102or the specific embodiment of the OCD 301.

In another general example embodiment, a data enablement system (alsoherein called the data enablement platform) is provided that includescloud computing servers that ingest audio data originating from one ormore user devices, the audio data comprising at least oral conversationof one or more users, and the cloud computing servers configured toapply machine learning computations to extract at least content andsentiment data features.

There are also data science servers that are in data communication withthe cloud computing servers and an external artificial intelligencecomputing platform. The data science servers also include a library ofdata science algorithms used to process the content and sentimentfeatures. In other words, the data science algorithms may also bespecific to given search bots or behavior bots. The data science serversoutput response data to the cloud computing servers, the response databeing in response to the audio data. Subsequently, the cloud computingservers format the response data into an audio data format playable by agiven user device, and transmit the formatted response data.

It will be appreciated that any module or component exemplified hereinthat executes instructions may include or otherwise have access tocomputer readable media such as storage media, computer storage media,or data storage devices (removable and/or non-removable) such as, forexample, magnetic disks, optical disks, or tape. Computer storage mediamay include volatile and non-volatile, removable and non-removable mediaimplemented in any method or technology for storage of information, suchas computer readable instructions, data structures, program modules, orother data. Examples of computer storage media include RAM, ROM, EEPROM,flash memory or other memory technology, CD-ROM, digital versatile disks(DVD) or other optical storage, magnetic cassettes, magnetic tape,magnetic disk storage or other magnetic storage devices, or any othermedium which can be used to store the desired information and which canbe accessed by an application, module, or both. Any such computerstorage media may be part of the servers or computing devices oraccessible or connectable thereto. Any application or module hereindescribed may be implemented using computer readable/executableinstructions that may be stored or otherwise held by such computerreadable media.

It will be appreciated that different features of the exampleembodiments of the system and methods, as described herein, may becombined with each other in different ways. In other words, differentdevices, modules, operations, functionality and components may be usedtogether according to other example embodiments, although notspecifically stated.

The steps or operations in the flow diagrams described herein are justfor example. There may be many variations to these steps or operationsaccording to the principles described herein. For instance, the stepsmay be performed in a differing order, or steps may be added, deleted,or modified.

The GUIs and screen shots described herein are just for example. Theremay be variations to the graphical and interactive elements according tothe principles described herein. For example, such elements can bepositioned in different places, or added, deleted, or modified.

It will also be appreciated that the examples and corresponding systemdiagrams used herein are for illustrative purposes only. Differentconfigurations and terminology can be used without departing from theprinciples expressed herein. For instance, components and modules can beadded, deleted, modified, or arranged with differing connections withoutdeparting from these principles.

Although the above has been described with reference to certain specificembodiments, various modifications thereof will be apparent to thoseskilled in the art without departing from the scope of the claimsappended hereto.

The invention claimed is:
 1. A distributed computing system comprisingserver machines that form a data enablement platform, the dataenablement platform comprising: a plurality of data collectors thatstream data over a message bus to a streaming analytics and machinelearning engine; a data lake and a massive indexing repository forrespectively storing and indexing data; a behavioral analytics andmachine learning module; multiple application programming interfaces(APIs) to interact with the data lake and the massive indexingrepository, and to interact with multiple applications; wherein themultiple applications comprise multiple command cards, and a givencommand card is specific to a given user and the given command cardcomprises a directive module that stores a given directive, a memorymodule, one or more search bots that search for data that is relevant tothe given directive, and one or more behavior bots that process the dataobtained by the one or more search bots according to one or morebehavioral attributes of the given user.
 2. The computing system ofclaim 1 wherein the one or more behavior bots apply one or moreartificial restrictions to the data obtained by the one or more searchbots, and the one or more artificial restrictions are associated withthe one or more behavioral attributes of the given user.
 3. Thecomputing system of claim 1 wherein the one or more behavior botscombine data from a different topic to the data obtained by the one ormore search bots, and the data from the different topic is associatedwith the one or more behavioral attributes of the given user.
 4. Thecomputing system of claim 1 wherein the one or more search bots searchfor the data that is relevant to the given directive taking into accountone or more criteria related to the given directive; and the one or morebehavior bots assess one or more attributes of the data obtained by thesearch bots; wherein the one or more attributes of the data areunrelated to the given directive and are associated with the one or morebehavioral attributes of the given user.
 5. The computing system ofclaim 4 wherein the data obtained by the search bots is ranked accordingthe one or more attributes.
 6. The computing system of claim 1 whereinthe one or more behavior bots generate a voice presentation of the dataobtained by the one or more search bots in a voice that is responsive toa current behavioral attribute of the given user.
 7. The computingsystem of claim 6 wherein if the given user is detected to be in a sadmood or agitated mood, the one or more behavior bots generate the voicepresentation in a cheerful voice.
 8. The computing system of claim 6wherein if the given user is detected to be in a sad mood or agitatedmood, the one or more behavior bots generate the voice presentation in avoice that is familiar to the given user.
 9. The computing system ofclaim 6 wherein if the given user is detected to be in a busy orconcentrated mood, the one or more behavior bots generate the voicepresentation in a neutral voice.
 10. The computing system of claim 6wherein if the given user is detected to be in a busy or concentratedmood, the one or more behavior bots generate the voice presentation withwords that are spoken quickly.
 11. The computing system of claim 6further comprising one or more synthesized voice libraries, wherein eachof the one or more synthesized voice libraries comprise voice parameterfeatures of one or more corresponding people, and the one or morebehavior bots select at least one of the synthesized voice libraries togenerate the voice presentation.
 12. The computing system of claim 11wherein the voice parameter features comprise two or more of: tone;frequency; loudness; rate at which a word or phrase is said; phoneticpronunciation; lexicon; syntax; articulation; rhythm; and melody. 13.The computing system of claim 1 wherein, in the given command card,there are multiple search bots that are each assigned to a differentdata source, and the multiple search bots all search for data that isrelevant to the given directive of the given command card.
 14. Thecomputing system of claim 1 wherein the plurality of data collectorscomprising multiple search bots from the multiple command cards, andeach search bot generates a separate and parallel stream of data. 15.The computing system of claim 1 wherein the streaming analytics andmachine learning engine comprises the multiple search bots from themultiple command cards, and each search bots generates a parallel streamof data.
 16. The computing system of claim 15 wherein a first search bothas a first set of streaming analytics modules to process data collectedfrom a first data source, and the second search bot has a second set ofstreaming analytics modules to process data collected from a second datasource.
 17. The computing system of claim 1 wherein the behavioralanalytics and machine learning module comprises multiple behavior botsthat monitor user interaction data via a user interface, and furtherprocess the data to obtain personalized data.
 18. The computing systemof claim 17, wherein the memory module of the given command card isprovisioned on the data lake and the massive indexing repository, andthe personalized data is stored in the memory module of the givencommand card.
 19. The computing system of claim 17 wherein data links tosource data and data summarizations of the personalized data are storedin the memory module of the given command card.
 20. The computing systemof claim 1 wherein the given directive is obtained by processing speechdata of the given user.
 21. The computing system of claim 20 wherein thespeech data is derived from data recorded by a microphone that detectsthe given user's voice.
 22. The computing system of claim 20 wherein thespeech data is derived from data recorded by at least one of a brainsignal sensor, a muscle signal sensor and a nerve signal sensor that ison the given user.
 23. The computing system of claim 1 wherein the oneor more behavioral attributes of the given user are derived from userdata obtained using at least one of a brain signal sensor, a musclesignal sensor, and a nerve signal sensor that is on the given user. 24.The computing system of claim 1 wherein the one or more behavioralattributes of the given user are derived from user data obtained usingat least a microphone.
 25. The computing system of claim 1 wherein theone or more behavioral attributes of the given user are derived fromuser data obtained using at least a camera.